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very  similar  commercial  manuals  is  not  warrented.  A  better  course  of  action  seems 
to  be  to  insist  that  DMWRs  be  written  and  updated  against  usability  standards , 
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COMPARING  DEPOT  MAINTENANCE  WORK  REQUIREMENTS 
(DKVR)  AND  COMMERCIAL  OVERHAUL  MANUALS 
FOR  EASE  OF  USE 


ABSTRACT 


The  Depot  Systems  Command  <DESCOM> ,  a  division  of  the 
Army  Material  Command  <AMC) .  has  sought  to  increase  the 
efficiency  of  the  Army  Depot  system  and  reduce  its  costs, 
and  has  targeted  its  overhaul  manuals  (the  Depot  Maintenance 
Work  Requirements  (DMVRs) )  for  termination.  It  is  DESCOM’ s 
contention  that  the  DMVR  does  not  compare  favorably  with 
other  forms  of  technical  overhaul  manuals,  particularly  not 
with  the  best  forms  of  overhaul  manuals  of  the  commercial 
world,  and  it  wants  to  adopt  these  "other”  forms.  However, 
DESCOM  has  not  specifically  stated  what  is  desirable  or 
undesirable  in  manuals,  nor  what  is  bad,  with  respect  to 
DMVR  usability,  or  what  is  good,  with  respect  to  commercial 
manual  usability. 

This  study  provided  a  locus  of  verifiable  data  and 
related  research  to  act  as  a  source  to  answer  some  of  these 
questions.  It  employed  six  common  manual  usability  tests  to 
compare  a  representative  sample  of  overt  iuI  manuals  from  the 
commercial  and  the  Depot  sectors.  The  findings  of  the  tests 
indicated  that  there  are  no  significant  differences  in  the 
two  manual  sets. 

One  implication  of  the  findings  is  that,  since  DMWRs  are 
not  significantly  different  from  commercial  manuals,  trading 
DMVRs  for  commercial  manuals  (technical  data  packages)  may 
be  simply  more  costly  than  beneficial.  Another  implication  for 
DESCOM  is  that  the  practice  of  editing  commercial  manuals 
(technical  data  packages)  to  make  them  into  DMVRs  might  not 
necessarily  add  value  to  DMVRs  or  the  Depot  system.  Whether 
DMVRs  are  scrapped  or  not,  however,  the  findings  and 
principles  outlined  in  this  study  seem  to  represent  good 
standards  (at  low  costs)  for  any  technical  manual  to  have. 
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CHAPTER  I 


INTRODUCTION 

The  US  Army  is  well  known  as  an  organization  that  has 
the  capacity  to  be  both  a  very  big  business  as  well  as  a 
natural  monopoly.  Vho  in  the  civilian  world,  for  instance, 
has  battle  tanks  to  develop,  maintain,  and  operate? 

Millions  are  appropriated  in  advance  each  year  by  Congress 
to  operate  the  Army,  and  in  many  cases,  the  Army  has  the 
only  capability  to  earn  those  dollars.  Competitive  bids  by 
parties  outside  the  military  are  encouraged,  but  competing 
to  get  a  piece  of  the  appropriation  has  been  difficult. 

Therefore,  The  Office  of  Management  and  Budget  has  set  a 
policy  for  Department  of  Defense  that  commercial  industry 
will  be  the  repair  sector  of  choice,  if  possible,  to  do 
organic  (military)  workload.  (35:A-2>*  This  poses  little 
concern  for  the  military  when  it  leels.  it  is  competitive 
with  the  commercial  sector.  However,  there  is  evidence  that 
the  military  has  lost  some  competitiveness:  the  commercial 
sector  is  growing  in  relation  to  the  military  sector.  (11:1-2) 


*  A  numbered  bibliography  is  used  in  this  study.  The  first 
number  identifies  the  reference  number  in  the  bibliography. 
The  second  number  identifies  the  page  of  the  reference. 
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Military  equipment  repair  Depots  are  prime  examples  of 
military  establishments  which  compete  for  work  with 
contractors  in  both  peace  and.  war.  Depots  have  a 
competitive  advantage  still,  because  a  set  percentage  split 
<70/30)  for  Depot  vs  contractor  workload  levels  <31:E-9c> 
exists.  This  maximum  ’’split”  means,  essentially,  that  a 
contractor  may  bid  for  work  destined  for  the  Depot  only  if 
the  work  is  in  excess  of  the  Depot’s  capacity  to  repair. 
Eighty-five  percent  facility  capacity  utilization  is 
recognized  by  US  industry  as  the  minimum  for  efficiency, 
while  present  Depot  capacity  is  slightly  more  than  50%,  on 
average.  <36:3-2) 

However,  Depots  still  believe  that  present  and/or  future 
business  is  in  jeopardy.  They  are  seeking  ways  to  cut  any 
cost  to  maintain  a  margin  of  safety.  One  such  attempt  seeks 
to  find  the  relative  casts/benefits  of  Depot  equipment  repair 
manuals  compared  to  those  of  the  private  sector,  in  order  to 
use  the  better  manual.  The  initiative  is  called  ’’Best 
Commercial  Practices"  <BCP) ,  and  is  discussed  in  the  section 
of  this  study  titled  "Background”. 

Depot  Systems  Command  <DESC0M) ,  a  higher  headquarters 
over  the  various  Army  Depots,  feels  that  its  manuals,  the 
Depot  Maintenance  Work  Requirements  <DMVR) ,  compare  badly  to 
manuals  of  the  commercial  sector.  It  feels  that  this  weak¬ 
ness  contributes  to  higher  costs  for  repair.  DESCOM  feels 
that  the  DMVRs’  intrinsic  weaknesses  cause  such  things  as 

1.  less  responsive  repair  to  customers 

2.  difficulty  in  being  updated 

3.  poor  ability  to  match  competitors’  cost,  which 
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results  in  poor  capacity  utilization  of  depots 

4.  poor  ability  to  natch  competitors’  use  of  state  of 
the  art  repair  processes,  which  results  in  workload  which 
only  requires  ’’old  tecnnology”  repair  capability.  <40:1) 

These  ’’intrinsic  weaknesses"  hamper  depot 
competitiveness  for  repair  of  military  equipment.  Military 
equipment  developers,  required  to  produce  equipment  under 
budget,  opt  to  develop  no  Depot  maintenance  future  for  a 
particular  piece  of  equipment,  rather  than  do  so  and  be 
faced  with  the  prospect  of  going  over  budget  thereby.  <40:1) 
DESCOM  requires  these  developers  to  provide  either  a  DMVR  or 
great  quantities  of  technical  data  before  Depots  may  perform 
work.  Since  DMVRs  are  very  expensive,  some  developers  opt 
not  to  develop  them,  in  favor  of  providing  the  ’’technical 
data*’  instead.  <40:1)  Without  work  for  the  Depots,  Depots 
stagnate,  and  could  eventually  be  scrapped. 

DESCOM  feels  that  this  a  vicious  circle,  and  could  be 
short-circuited  if  DMVRs  were  intrinsically  better  manuals. 
To  DBSCOM,  the  DMVR  is  ”a  major  reason  for  the  current 
decline  in  organic  Depot  systems  competitiveness  for 
repair/overhaul  work  with  respect  to  contract  Sources  of 
Repair."  <40:1)  The  BCP  initiative  is  designed  to  correct 
this  decline  by  requiring  that  all  repair/overhaul  be  done 
in  accordance  with  other  forms  of  technical  data  rather  than 
with  the  DMVRs. 

The  Commander,  AMC,  has  applauded  this  effort,  and 
mandated  that  a  pilot  program  to  test  the  concept  <being 
conducted  at  Anniston  Army  Depot)  be  immediately  expanded. 

He  has  requested  that 
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each  MSC  identify  at  least  one 
commodity,  which  presently  requires  the 
use  of  DMVR,  for  establishment  of  a 
pilot  program  to  explore  the 
effectiveness  of  [this  initiative]. 

<39: 1) 

One  potential  problem  that  may  inhibit  the  pilot 
program  and  future  operations  of  the  depots  is  that  DESCOM 
has  not  published  verifiable  findings  establishing  what  is 
specif ically  ••wrong”  with  DMVRs,  or  what  is  specifically 
"right**  with  commercial  manuals.  For  instance,  no  data 
axists  comparing  BXVRe  with  commercial  manuals  with  respect 
to  ease  of  use,  or  the  ability  to  be  updated,  or  other 
similar  quantifiable  measures  for  comparison.  It  is  not 
possible  to  evaluate  something  until  it  is  defined 
objectively. 

Purpose  of  Study 

The  purpose  of  this  study  is  to  compare  classes  of 
equipment  overhaul  manuals  of  the  US  Army  Depot  maintenance 
system  with  corresponding  classes  of  manuals  of  the  US 
commercial  equipment  maintenance  system  to  determine  if 
Depot  overhaul  manuals  are  relatively  more  difficult  to  use. 

Analysis  of  the  Problem  <subproblems> 

In  order  tc  determine  the  relative  ease  of  use  of  Depot 
versus  Commercial  Overhaul  manuals,  the  following 
subproblems  must  be  addressed. 

The  first  subproblem.  The  first  subproblem  is  to 
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determine  the  relative  simplicity  of  the  manuals,  as  In 
their  ease  of  readability. 

The  second  subproblem.  The  second  subproblem  is  to 
determine  the  relative  number  of  Illustrations  per  manual 
between  the  manual  sets  (commercial  vs  DHVR) . 

The  third  subproblem.  The  third  subproblem  is  to 
determine  the  frequency  of  use  of  any  aids  to  understanding 
in  the  texts,  such  as  cues,  marginalia,  and  underlining. 

Hypotheses 

The  First  Hypothesis.  The  first  hypothesis  is  that 
there  is  no  significant  difference  between  the  commercial 
and  Depot  manuals'  readability. 

The  Second  Hypothesis.  The  second  hypothesis  is  that 
there  is  no  significant  frequency  difference  between  the 
commercial  and  the  Depot  manuals'  illustrations. 

The  Third  Hypothesis.  The  third  hypothesis  is  that 
there  is  no  significant  difference  between  the  commercial 
and  Depot  manuals'  relative  practice  of  using  aids  to 
understanding  in  the  text,  aids  such  as  cues,  marginalia, 
and  underlining. 

Assumptions 

The  first  assumption.  Depot  and  commercial  mechanics 
and  technicians  are  equally  skilled  in  the  use  of  repair 
standards,  criteria,  and  procedures,  from  repair 
publications. 
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The  second  assumption.  Depot  and  commercial  mechanics 


and  technicians  are  equally  motivated  to  use  repair 
standards,  criteria,  and  procedures  from  repair 
publications. 

The  third  assumption.  Depot  and  commercial  mechanics 
and  technicians  can  equally  apply  repair  standards,  criteria 
and  procedures  from  repair  publications  to  produce  high 
quality  products. 

The  fourth  assumption.  The  modern,  generally  accepted 
readability  criterion  for  published  matter  indicate  well  the 
reading  grade  level  of  a  text. 

The  fifth  assumption.  Comparing  general  classes  of 
equipment  repair/overhaul  manuals  (for  instance,  commercial 
and  military  diesel  engine  overhaul  manuals),  will  not 
introduce  significant  bias  about  comparative  text 
readability,  number  of  illustrations,  or  use  of  textual  aids 
to  understanding. 

The  sixth  assumption.  Picking  only  three  standard 
usability  criteria,  e.g. ,  reading  grade  levels,  heuristic 
(cue  and  illustration  frequency),  and  user  testing,  will  be 
sufficient  so  that  an  unbiased,  reasonable  difference  could 
emerge  to  be  evaluated. 

The  seventh  assumption.  The  perception  that  the  DKVR  is 
hard  to  use  exists  with  those  workers  who  actually  use  the 
manual  as  well  with  those  workers  at  the  higher,  DESCOK 
levels,  who  wrote  the  BCP  initiative. 

The  eighth  assumption.  The  illustrations,  cues,  and 
words  counted  and  compared  represent  correct  repair 
technology. 
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Scope  and  Limitations 


This  study  includes  manual  comparisons  for  only  three 
sets  of  items  of  equipment  used  by  both  the  Army  and  the 
civilian  sector.  These  three  items  represent  the  test 
population  because  they  seem  to  typify  best  the  Army  Depot 
commodities  that  are  overhauled  (aviation,  ground  mobility, 
communication/electronics) . 

This  study  focuses  on  the  depot /commercial  type  of 
maintenance  best  relating  to  "reliability  centered" 
maintenance  (replace  as  needed)  as  opposed  to  "strict 
overhaul"  (replace  all  parts). 

This  study  does  not  evaluate  any  other  variables  of 
equipment  repair  such  as  test  and  diagnostic  equipment, 
tools,  and  facilities. 

This  study  does  not  explore  the  ability  of  mechanics/ 
technicians  to  repair  items  by  "experience",  "feel"  or 
"intuition",  as  opposed  to  using  a  manual. 

This  study  does  not  attempt  to  validate  the  procedures 
for  developing  manuals,  or  the  constraints  under  which 
manual  writers  write,  or  the  qualifications  of  manual 
writers  to  write  manuals. 

Although  this  study  is  limited  in  its  scope,  it  provides 
a  body  of  data  which  AMC  (the  proponent  for  writing  DKVRs) 
does  not  currently  have,  which  could  be  used  to  improve 
DXVRS  of  all  kinds.  Improved  DMVRs  would  contribute  much  to 
reducing  costs,  increasing  readiness,  and  improving  the 
working  conditions  of  Depot  employees. 


-7- 


Definition  of  Tsmw 


The  following  terms  are  used  throughout  this  paper. 

They  are  words  and  phrases  not  generally  used  outside  the 
realms  of  depot  or  commercial  repair,  and  are  important  to 
know  to  understand  this  study. 

Depot  Maintenance  Activity.  A  Depot  is  an  industrial  type 
facility  established  to  perforin  Depot  level  maintenance  on 
weapon  systems,  equipment,  and  components  of  the  military. 
The  term  includes  DoD  installations  and  commercial 
contractors. 

Depot  Maintenance  Vork  Requirement  <DMVR> .  DM VR  is  the  name 
of  the  maintenance  manual  for  the  military  Depots. 

Industrial  Maintenance  Support.  This  support  is  Depot 
workload,  meaning  maintenance  performed  on  materiel  after 
its  withdrawal  from  custody  of  the  using  military  command. 

Maintenance  end  item.  An  end  item  is  a  final  combination  of 
assemblies,  components,  and  parts  that  performs  a  major 
operational  function. 


Background 


The  Department  of  Defense  uses  repair  Depots  to  overhaul 
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equipment  in  each  of  the  principal  sister  defense  services, 
Air  Force,  Navy  and  Army.  The  US  Army  Depot  maintenance 
program  is  typical,  and  is  directed  and  workloaded  by  the 
Depot  Systems  Command  <DESCOIC)  of  the  Army  Materiel  Command 
(AMC) ,  through  individual  commodity  commands  called  Major 
Subordinate  Commands  <MSCs> . 

Overall  Depot  maintenance  policy  is  formed  by  DESCOM.  It 
designates  primary  and  secondary  repair  facilities  for  an 
item,  and  collects  management  and  physical  data  about  each 
Depot  and  forwards  this  through  AMC  to  Army  Headquarters. 
Examples  of  this  data  important  to  this  study  are  cost  and 
timeliness  statistics  from  the  repair  of  items  of  equipment. 

The  MSCs  serve  as  item  managers  for  the  Army’s  classes 
of  weapons  systems  programs,  identifying  depot  level 
maintenance  requirements,  supporting  depots  and  field 
commands  with  their  commodity’s  repair  parts,  and  making  the 
decision  whether  a  Depot  Maintenance  Work  Requirement  (DMVR) 
will  be  required  for  a  developing  piece  of  equipment.  Many 
new  items  do  not  have  DMVRs  because  the  items  are  too  new  to 
require  overhaul,  so  the  writing  of  the  DMVR  is  postponed  to 
a  later  date.  In  most  cases,  if  an  item  has  no  DMVR,  the 
item  developer  has  decided  that  it  does  not  need  one. 
Examples  of  equipment  without  DMVRs  are  the  1  1/4  ton 
Commercial  Utility  Cargo  Vehicle  and  M915  diesel  engine. 

Depot  facilities  are  fixed,  industrial-type  buildings 
equipped  with  industrial  tools  and  test  equipment.  Each 
Depot  may  be  workloaded  with  hundreds  or  thousands  of 
different  line  items  of  equipment  to  repair.  Depots 
generally  repair  only  Army  items  and  subcontract  backlog  to 
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commercial  repairer  only  when  backlog  exceeds  capacity. 

Individual  depots  perform  the  highest  levels  of 
maintenance  repairs  within  the  Army  system.  Skilled 
Department  of  the  Army  <DA)  civilian  mechanics,  materiel 
workers  and  handlers,  technicians  and  engineers  do  the 
production  planning,  controlling,  quality  assurance  and 
other  processes  to  render  unserviceable  equipment  to  a  "like 
new"  condition.  In  the  current  three-level  army  maintenance 
system  <Unit  level,  Direct/General  support,  Depot),  the 
Depot  may  perform  each  level  as  necessary  and  allowable 
under  its  program. 

Equipment  repaired  in  the  Depot  comes  from  owning  units, 
but  that  same  piece  of  equipment  generally  does  not  return 
to  that  unit.  It  is  turned  in  to  the  supply  system  for 
general  distribution.  Additionally,  there  are  no 
differences  in  the  final  application  of  equipment  repaired 
in  the  Depot  and  commercial  scenario.  The  equipment 
repaired  in  either  sector  does  not  necessarily  go  to 
reserve/Uat ional  Guard  or  active  duty  units.  Therefore, 

MSCa  do  not  communicate  specific  requirements  differences  in 
materials,  repair  parts,  or  processes  of  repair  to  manual 
writers  because  of  the  sector  of  equipment  repair. 

DESCOM  believes  that,  when  DHVRs  do  not  exist, 
competitiveness  of  the  Depot  automatically  decreases  since 

equipment  developers  will  not  contact  Depots 
for  cost  estimates  on  repair  of  their 
equipment  when  they  know  no  DMVR  has  ever 
been  developed,  in  the  mistaken  belief  that 
all  Depot  work  must  be  done  by  DXWR.  <40: 1) 

Actually,  Depots  have  done  Depot  level  work  without  a 


DMVR.  <40: 1)  However,  existing  DMVRs  must  be  used, 

regardless  of  better  manuals  which  may  exist,  <40:1)  One 

problem  with  not  having  a  DMVR  is  that  depots  are  in  that 

case  very  reluctant  to 

provide  cost  estimates  to  equipment 
developers  for  equipment  without  a  DMVR, 
because  they  hesitate  to  provide  a 
questionable  (baseless)  estimate  fearing  they 
will  not  be  able  to  refine/  renegotiate  after 
gaining  enough  experience  to  develop  a  more 
realistic  unit-funded  cost,  <40: 1) 

Depots  operate  on  a  cost  basis.  They  are  programmed  to 
repair  a  certain  number  of  items  based  on  Army  need  and 
individual  Depots  manhours  available.  Army  needs 
essentially  dictate  the  number  of  manhours  available, 
constrained,  of  course,  by  budget  ceilings.  Depots  must 
operate  within  that  budget  and  produce  the  required  number 
of  repaired  items  to  military  specifications. 

Commercial  maintenance  facilities,  on  the  other  hand, 
operate  on  a  profit  basis.  They  cut  costs  any  way  they  can, 
and  change  as  often  as  they  must  in  order  to  stay  solvent. 

As  long  as  benefits  outweigh  liabilities,  commercial 
concerns  will  spend  money  to  make  money.  This  is  important 
to  keep  in  mind  in  the  subsequent  discussions  about 
commercial  technical  manuals.  Commercial  concerns  tend  to 
buy  manuals  which  allow  the  largest  profit  from  operations 
rather  than  follow  a  set  format  (such  as  the  Army  does). 

Commercial  maintenance  establishments  are  essential  to 
the  effective  conduct  o-f  war,  because  only  the  commercial 
sector  can  afford  to  run  the  number  of  facilities  needed  to 
repair  the  volume  of  equipment  that  a  modern  war  would 
destroy. 


Conve.^ely,  Depots  are  also  essential  to  the  effective 
conduct  of  war,  because  their  advantage — their  snail  size — 
allows  then  to  more  easily  change  operations  quickly  during 
crucial  mobilization  time  frames.  They  are  also  known  to 
have  a  relatively  lower  occurrence  of  default/nonconformance 
than  the  commercial  sector.  <36:3-3)  Depots  are  therefore 
designed  to  operate  under  set  procedures  at  all  times  so 
that  at  any  time  they  can  transform  without  undue  cost  and 
time  to  mobilize  the  Army. 

Peacetime  planners,  paid  to  cut  costs,  tend  to  forget 
this  wartime  mission  focus  of  Depots  in  favor  of  maximizing 
peacetime  cost  concerns.  They  try,  for  Instance,  to  lower 
the  high  standards  of  the  DMVR  in  order  to  cut  costs.  Since 
Depots  operate  on  a  cost  basis,  they  place  themselves  at 
risk  in  times  of  recession  <escalating  costs)  to  become 
uncompetitive  with  respect  to  the  commercial  sector.  Thus 
negatively  compared,  equipment  developers  opt  not  to  Include 
depots  in  maintenance  plans,  and  Depot  facilities  and  skills 
become  increasingly  underused.  Thus,  military  planners  of 
the  DMVR  must  maintain  an  acceptable  balance  in  a  tradeoff 
between  sometimes  divergent  goals. 

To  assume  a  fair  and  equitable  comparison  between 
contractor  and  depot  bids  for  repair  workload,  The 
Department  of  Defense  <DoD)  has  mandated  that  ” in-house  cost 
estimates  must  be  based  on  the  same  scope  of  work  [as  for 
contractors]."  (30:7-5)  This  is  important  to  the  research 
topic,  because  it  tends  to  ensure  that,  whatever  the 
application  of  Depot /commercial  repair,  work  within  the  DoD 
will  be  based  on  similar  scopes  of  work. 
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The  Depots  are  facing  a  "technological  revolution" 
(10:1-2),  meaning  new  technologies  exist  for  virtually  every 
aspect  of  Depot  life.  The  many  new  materials  and  better 
item  reliability  is  changing  the  workloads  of  Depots,  so 
Depots  need  to  modernize,  simplify,  and  become  more 
competitive  with  respect  to  commercial  sources  of  repair. 

The  Army  most  often  contracts  the  development  of  DJCVR 
writing  to  the  commercial  sector.  Very  few  DKVRs  are 
written  by  the  military  itself.  This  is  why  these 
publications  run  a  risk  of  being  developed  in  a  potential 
commercial  vacuum.  (8:2-18)  That  is,  coordination,  data 
input,  and  verification  can  become  cursory  between  a  widely 
separated  commercial  and  Depot  world.  This  can  increase 
manual  cost  and  decrease  effectiveness  because  of  the 
distances  over  which  coordination  must  be  accomplished. 
DESCOM  believes  that  "the  adoption  of  existing  technical 
data  C BCP]  will  be  less  costly  than  development  of  the 
formalized  DMVR"  (36:3-12),  ostensibly  due  in  part  to  the 
fewer  coordination  difficulties. 

The  Army  has  long  worked  to  improve  the  clarity  of  its 
manuals.  Indeed,  this  has  been  the  focus  of  research  and 
development  efforts  for  over  35  years.  (45:70)  Traditional 
maintenance  manuals  have  had  a  "staid  style,  tedious  and 
complex  language,  and  a  lack  of  illustrated  procedures" 
(47:26)  making  them  difficult  to  use. 

DESCOM  believes  that  commercial  manuals  are  easier  to 
use,  and  cheaper  to  write  and  update  since  they  are  not 
constrained  by  governmental  intervention  or  the  needs  of 
military  application.  DMVRs  are  "rigidly  structured. . . 
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C and]  many  original  DMVRs  are  inadequate  and  must  be 
rewritten  by  Depot  engineers  and  technical  writers”.  (37:1) 

Depots  have  a  publications  writing  section  that  updates 
DMVRs.  This  section  is  staffed  by  proven  technicians, 

<not  necessarily  proven  writers)  who  design,  update,  and 
change  DMVRs.  <8:2-25)  One  of  their  main  functions  is  to 
take  existing  tech  data  packages  developed  by  commercial 
sources,  and  modify  them  to  conform  with  the  DMVR  format 
spec i f icat ion . 

One  DMVR  manual  can  cost  over  3150,000,  with  each  page 
costing  between  $250  and  $300  to  write.  (8:2-14)  There  are 
over  5,300  DMVRs,  averaging  100  pages  per  manual,  and 
updating  them  therefore  is  expensive.  It  has  been  estimated 
that  to  update  the  (1984)  DMVR  library  to  the  latest 
military  specification  requirements  will  cost  $95  million. 
<8:2-35) 

Significance  of  the  Study 

US  Army  DESC0M  and  its  depots  have  been  faced  with  both 
shrinking  market  share  and  shrinking  budgets.  The  1990-1991 
Army  budget  has  likewise  sustained  a  $1.8  billion  reduction 
below  that  of  the  previous  year,  with  the  Army  being 
required  to  operate  at  1989  spending  levels.  <50: Cover 
story)  Additionally,  Depot  0MA  resourcing  will  take  a  $102.7 
million  growth  cut  over  the  next  5  years.  <36:3-1) 

The  proportion  of  cantract/depot  repair  work  appears  to 
be  shifting,  with  an  apparent  loss  of  market  share  for  the 
Depots.  A  modernization  move  by  DESCOM,  to  reduce  that 
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apparent  market  imbalance,  is  to  improve  its  repair  manuals. 
Its  proposal  is  to  use  its  $200  million  DMVR  technical 
library  less,  in  favor  of  other  technical  data  that  it  must 
develop,  because  it  feels  (but  has  not  yet  substantiated) 
that  its  DMVR  is  hard  to  use  and  update.  A  potential 
problem  for  DESCOM  is  that  it  may  develop  new  technical  data 
packages  which  contain  the  same  problems  that  the  DMVRs 
have,  unless  it  finds  out  what  the  problem  areas  are  first. 

This  study  will  seek  to  find  specific  reasons  why  the 
DMVR  is  purportedly  harder  to  use  than  comparable  commercial 
technical  data.  These  data  could  be  used  by  DESCOM  to 
improve  its  manuals,  rather  than  resort  to  throwing  them  out 
and  starting  fresh.  Diagnostic  upgrading,  after  all,  is 
what  each  Depot  normally  does  to  each  piece  of  equipment  it 
overhauls — to  fix  it,  rather  than  simply  to  throw  it  out  and 
buy  a  new  item. 

Outline  For  The  Remainder  of  the  Study 

The  remainder  of  this  study  consists  of  four  additional 
chapters.  Chapter  II  provides  a  review  of  the  literature 
and  regulations  that  relate  to  this  study.  Chapter  III 
describes  the  procedures  selected  for  collecting  and 
evaluating  the  data  needed  to  solve  the  problem.  Chapter  IV 
presents  the  results  of  study,  inferred  from  the  findings 
and  data  collected.  Chapter  V  contains  a  summary  of  the 
study's  proceedings,  recommendations  for  employing  the  data 
collected,  and  conclusions  drawn  as  a  result  of  the  study. 
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CHAPTER  II 


RELATED  LITERATURE 

This  chapter  contains  summaries  of  the  more  significant 
literature  about  "the  ease  of  use”  of  the  Depot  Maintenance 
Work  Requirement  (DMVR) .  It  also  discusses  commercial 
repair  practices  and  some  characteristics  of  commercial 
overhaul  manuals.  Subsections  of  this  chapter  are; 
developing  military  and  commercial  manuals,  previous 
evaluations  of  maintenance  manuals,  comparing  contract  vs 
organic  scenarios,  previous  evaluations  of  commercial 
manuals  in  the  Depots,  readability  analysis  in  technical 
manuals,  and  the  validity  of  the  tests  to  be  used.  This 
chapter  concludes  with  a  summary. 

Developing  Military  vs  Commercial  Manuals 

The  Department  of  Defense  <DoD>  has  developed  and  used 
quite  extensive  specifications  for  producing  its  technical 
manuals.  The  Defense  Logistics  Agency  <DLA>  identified  480 
different  specifications  for  writing  manuals.  <45: 75) 
However,  research  shows  that  these  specifications  have  not 
been  sufficient  alone  to  produce  usable  manuals. 
Specifications  ’’only  address  the  manual  as  a  product,  not 
how  to  produce  that  product."  <45:75)  It  is  as  important  to 
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make  a  technical  manual  readable  as  to  make  it  technically 
accurate. 

The  commercial  and  the  Depot  sectors  both  strive  to  have 
the  best  manuals  possible,  for  the  lowest  costs,  in  order  to 
reach  the  ultimate  goal  of  low  cost  and  quality  repair.  The 
need  for  cost  effectiveness  in  the  competitive  environment 
of  equipment  overhaul  is  high.  However,  no  one  has 
attempted  to  increase  effectiveness  of  repair  by  comparing 
Depot  to  commercial  manuals  to  try  to  grade  possible 
usability  differences. 

One  of  the  chief  Army  regulations  stipulating  manual 
development  specifications  is  Army  Regulation  <AR>  25-30, 

The  Army  Integrated  Publishing  and  Printing  Program,  dated 
March,  1989.  It  mandates  very  high  and  very  comprehensive 
standards  for  all  Army  publications,  with  entire  chapters 
devoted  to  central  concerns  such  as  ”  Writing  and  Revising 
Publications”,  ’’Preparing  tables  and  illustrations”,  and 
’’changes,  revisions,  reprints,  and  recisions.”  C22:  index) 

Information  of  particular  relevance  to  this  research 
from  this  regulation  are  the  tests  for  readability,  which 
establish  reading  grade  levels  (RGLs) .  The  Army  has  taken 
great  pains  to  require  that  its  manuals  be  specific  in 
content  as  well  as  readable. 

Specifications  for  the  content  and  format  of  Technical 
manuals  come  from  MIL-M-38784B  Military  Spec:  General  Style 
and  format  Requirements.  Among  other  things,  it  stipulates 
that  technical  manuals  will  have  readability  standards 
<32a:3.3.3),  illustration  standards  (32a: 3. 5. 7),  and 
illustration  scale  and  letter  point  size  minimums 
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(32a: 3.6. 1). 

Specifications  for  the  content  and  format  of  DICVRs  come 
from  JCIL-JC-63041C  Preparation  of  Depot  Maintenance  Work 
Requirements.  (33)  The  Army  is  so  serious  about  its  desire 
to  have  usable  manuals  that  there  is  even  a  specification 
for  the  specification,  and  the  final  draft  is  awaiting 

approval.  The  draft,  MIL-X-63041D.  clarifies  ambiguous 
passages  of  XIL-X-63041C.  improves  its  readability, 
amplifies  its  discussion  of  standards,  and  makes  ijt  more 
usable  for  writing  the  DJCVR. 

Specific  standards  for  preparing  manuals  for  the 
commercial  sector  exist,  but  vary  widely  as  to  content, 
depth  of  coverage,  and  style.  Commercial  manuals  are 
generally  written  by  different  publishers  with  different 
standards  (45:70).  This  is  relevant  because  it  seems  to 
suggest  the  philosophical  differences  between  commercial  and 
military:  the  military  is  more  authoritarian  in  its 
specification  and  values  consistency  in  its  manuals  across 
the  broad  spectrum  of  its  repair  facilities. 

Other  regulations  help  ensure  Army  publications  are  both 
standard  and  contain  information  covering  every  maintenance 
contingency.  AR  700-127  Integrated  Logistics  Support 
contains  detailed  plans  for  developing  maintenance  planning, 
test/measurement  and  diagnostic  equipment  needs,  and 
technical  data  for  equipment  being  developed.  It  thus  has  a 
indirect  but  significant  impact  on  the  DJCVR.  (25:3-10) 

AR  70-1,  Acquisition  Policy,  stipulates  at  what  period 
in  the  equipment  development  process  the  decision  must  be 
made  whether  DJCVR  will  be  required  for  the  piece  of 
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equipment.  <23:5-1)  In  theory,  the  earlier  a  DMVR  is 
planned  and  written  in  the  life  of  a  piece  of  equipment,  the 
better  the  manual  will  be. 

The  relevance  of  these  discussions  to  the  research  topic 
is  that  they  summarize  the  apparent  differences  between  the 
development  of  commercial  and  military  manuals.  These 
differences  need  to  be  taken  into  account  when  determining 
the  ease  of  use  of  the  DMVR,  because  knowing  the  manual 
development  philosophies  can  help  orient  judgements  about 
the  manuals  themselves. 

Previous  Evaluations  of  Maintenance  Manuals 


The  DMVR  has  undergone  many  evaluations  for  the  purpose 
of  determining  its  acceptability  as  a  maintenance  manual. 
These  evaluations  each  explore  different  aspects  of  the 
effects  of  DMVR  use  on  Depot  repair  management  and 
operations.  They  can  be  subdivided  into  two  categories: 
subjective,  and  objective,  evaluations.  The  ’’subjective 
evaluation”  is  the  Best  Commercial  Practices  initiative, 
already  discussed  in  the  introduction  and  background. 

The  "objective  evaluations”  are  those  from  the  Material 
Readiness  Support  Agency  <MRSA>,  various  Integrated 
Logistics  Support  (ILS)  offices,  and  The  General  Accounting 
Office  (GAO) .  Others  include,  a  commercial  engineering 
firm,  the  Naval  Postgraduate  School,  and  the  Air  College. 
These  each  are  bodies  of  verifiable  data  from  which  valid 
conclusions  can  be  drawn.  None  quantified  the  usability 
differences  between  Depot  and  commercial  manuals,  however. 
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The  purpose  of  the  XRSA  study  (1985),  was  to  "assess  the 
overall  efficiency  of  the  current  policies,  management 
structure,  and  operational  procedures  of  DMVR  management" 

<8: ill).  The  study  was  chartered  to  address  "ongoing 
improvement  actions. . .  and  quality  and  efficiency  of  Depot 
writing  groups.”  <8:  end  1) 

A  product  of  this  study  was  its  exhaustive  list  of 
recommendations  proposed  to  improve  the  DMVR,  based  on  the 
research  questions,  "Is  the  current  DMVR  system  cost 
effective,  and  does  it  make  sense?"  (8:1-1)  Overall,  the 
study  determined  that  "both  the  product,  ie,  DMVR 
publication,  and  the  management  of  the  DMVRs  ,  could  be 
improved."  <8: iv) 

notwithstanding  this  general  negative  comment,  many 
specific  comments  were  positive.  The  study  found  that  Depot 
writers/revisors  of  the  DMVR  were  not  trained  manual 
writers,  but  were  still  "producing  satisfactory 
DMVRs. .. which  require! dl  very  little  substantive  change  at 
DMVR  validation.”  (8:2-18)  Some  manuals  were  found  to  be 
out  of  date  (8:2-35),  but  regulations  and  guidance  for  the 
preparation  of  DMVRs  were  judged  to  be  "clear,  specific  and 
adequate,  in  general."  (8:2-53) 

The  relevance  of  this  study  to  the  research  topic  is 
that  it  suggests  that  many  faults  in  DMVRs  may  come  from 
forces  outside  itself,  ie,  from  writers  and/or  manual 
validators. 

A  Haval  Postgraduate  School  study  investigated  an 
apparently  excessive  cost  differential  between  electronics 
equipment  repair  done  at  different  Depots  in  1984.  (19:45) 
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The  researcher  tried  to  determine  what  contributed  to  these 


differentials.  A  finding  of  the  study  was  that  repair  and 
overhaul  standards  procedures  differed  at  each  Depot  for  the 
same  items.  <19:60) 

This  finding  is  important  to  this  research  study, 
because  it  shows  graphically  that  depots  without  DMVRs  can 
develop  repair  methods  which  are  of  lesser  or  greater  cost 
effectiveness.  It  supports  with  evidence  that  when  manuals 
are  allowed  to  be  written  based  on  best  practices  rather 
than  on  standard  guidance  (as  in  specifications),  cost 
differences  in  repair  of  similar  items  can  result.  This 
tends  to  indicate  that  using  manuals  designed  on  commercial 
practices  rather  than  MIL-M-63041C  can  have  negative  side 
effects. 

The  General  Accounting  Office  <GA0>  prepared  a  study  in 
1979,  titled  ’’Management  of  Maintenance  Manuals  in  DoD.  ”  The 
study  reported  that  maintenance  manuals  were  often  difficult 
to  use,  were  often  out  of  date,  and  were  inaccurate.  <12: i) 
The  study  concluded  that  in  many  cases  the  value  of 
maintenance  data  was  questionable  because  descriptions  in 
manuals  were  too  complicated,  complex  procedures  were  not 
explained  in  sufficient  detail,  and  necessary  information 
was  very  difficult  to  locate.  <12:5)  However,  the  GAO  also 
found  that  Army  management  of  DMVR  was  ’’best  DoD  wide  and 
good  enough  to  be  the  standard  for  all  services. . . in 
development,  coordination,  land  for  ensuring]  uniformness 
between  operators  manuals  and  maintenance  manuals."  <12:27) 
The  GAO  recommended  that  standardizing  the  use  of 
specifications  for  preparing  manuals  would  ease  the  burden 
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of  the  contractors  furnishirg  specifications  to  the 
services.  (12:28) 

The  relevance  of  this  study  to  the  research  topic  is 
that,  though  it  found  that  maintenance  manuals  were  hard  to 
use  in  general,  it  found  that  both  the  Army  management  and 
its  publication  specifications  were  good  and  beneficial. 
Therefore,  it  is  likely  that  Army  Depot  manuals  are  the  most 
usable  service-wide. 

The  US  Air  force,  in  conjunction  with  the  TRW 
Corporation,  did  a  study  in  1979  to  determine  why  many 
repair  techniques,  procedures,  and  allowable  limits  for 
repair  in  commercial  aircraft  manuals  were  so  different  from 
those  of  the  Air  force.  (16:1)  Of  all  the  many  thousands  of 
procedures  surveyed,  in  eight  sets  of  manuals  of  identical 
pieces  of  equipment  in  and  out  of  the  Depot,  only  twenty- 
three  significant  differences  were  found  overall.  (16:2) 

The  study  found  that  commercial  aircraft  engine  overhaul 
manuals  appeared  to  contain  a  larger  number  of  repair 
procedures  than  did  the  Air  Force  Technical  manuals,  and 
that  "differences  in  operating  environment,  inspection 
criteria,  and  overhaul  time  1 imi ts,  .  .  I  made)  a  commercial 
procedure  unsuitable  for  Air  Force  use."  (16:3) 

Additionally,  a  number  of  commercial  repair  manual 
procedures  were  considered  by  the  TRW  engineers  to  be  not 
economical,  not  sound  for  engineering,  or  merely  not  needed. 
(16:A-6)  Commercial  manuals  were  not  always  better  than 
military  manuals,  according  to  this  study.  Comparisons  of 
manual  usability  was  not  undertaken  by  this  study. 

An  ongoing  URSA  study  called  "Integrated  Logistics 
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Support  <ILS>  Lessons  Learned”  records  the  effects  of  DMVR 
usability  on  integrated  logistics  support.  The  following 
reports  are  of  particular  relevance  to  the  research  topic, 
because  they  have  recorded  the  actual  effects  of  poor  DMVR 
usability: 

DMVRs,  that  do  not  represent  the  latest  end 
item/ weapon  system  configuration  because  of 
configuration  changes  not  incorporated  into 
the  final  DMVR,  delay  in-house  overhaul. 

(13:7-7) 

Many  [ user — oriented]  problems  are  encountered 
when  the  Depots  are  not  involved  early  in  the 
development  of  contractor  prepared  DMVRs. 

(7:1) 

When  a  technical  data  package  has  never  been 
purchased  for  a  piece  of  equipment,  and 
equipment  arrives  in  a  depot  for  repair, 

Depots  must  employ  costly  reverse  engineering 
procedures  to  develop  technical  data.  (9-24) 

Elements  of  DESCOM  appeared  to  share  these  views.  DBSCOM 
circulated  letters  within  the  Army  Material  Command  (AMC) 
which  stated  that  commercial  maintenance  practices  (as 
evidenced  in  their  manuals)  in  general  "  incorporate!!  d] 
latest  state-of-the-art  processes,”  (37:1)  and  had  merit  of 
which  the  ”ef feet iveness.  ..[  should  be]  explored.”  (39:1) 
DESCOM  also  believed  ’’these  practices  would  probably  reduce 
cost  and  the  lead  time  necessary  to  develop  DMVRs.”  (41:1) 
The  relevance  of  these  claims  to  the  research  topic  is 
that  they  indicated  that  DMVRs  were  not  well  liked  by  their 
users.  As  these  claims  were  written  only  two  months  ago,  it 
seems  also  likely  that  the  same  attitudes  could  still  exist. 
And,  these  claims  must  have  some  validity,  since  they  came 
from  a  user .  A  user  is  generally  in  a  good  position  to 
render  a  reasonable  judgement  about  something  it  uses. 
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However,  since  this  user  (DESCOX)  provided  no  verifiable 
data  in  defense  of  its  opinions,  these  opinions  must  be 
classified  as  "subjective"  for  the  time  being. 

The  relevance  of  the  many  "objective  evaluations"  to  the 
research  topic  is  that  they  tend  to  show  that  DXVRS  and 
commercial  manuals  can  be  compared,  that  the  previous 
comparisons  have  been  for  aspects  other  than  ease  of  use, 
and  only  insignificant  differences  to  date  have  been  found 
in  the  technical  aspects  of  Depot  vs  commercial  manuals. 

Comparing  Contractor  vs  Organic  Scenarios 

The  fallowing  literature  constitutes  background  on  the 
differences  between  contractor  and  depot  (organic  military) 
scenarios,  so  it  is  indirectly  related  to  the  "ease  of  use 
of  the  DMVR."  However,  it  is  important  to  understand  in 
order  to  Judge  the  commercially  developed,  military  used 
DMVR. 

The  policy  of  the  US  government  regarding  contracting  is 
covered  in  the  Federal  Acquisition  Regulation  (FAR).  It 
prescribes  policies  and  procedures  for  use  in  acquisitions 
of  commercial  or  industrial  products  and  services  based  on 
Office  of  Management  and  Budget  (OMB)  Circular  ¥o.  A-76.  It 
provides  guidance  to  DoD  and  the  Army  about  all  aspects  of 
giving  or  receiving  of  goods  or  services,  including  the 
preparation  of  army  manuals. 

The  FAR  specifies  that  work  performance  standards,  upon 
which  much  of  government  technical  manuals  are  based,  "must 
ensure  a  comparable  level  of  performance  for  both  Government 
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and  contractor  and  a  common  basis  for  evaluation." 

<32:7-4,1)  This  is  relevant  to  this  research  topic  because 
it  ensures  that  no  one  will  arbitrarily  reduce  the  required 
steps  in  a  repair  procedure,  solely  to  undercut  a 
competitor’s  bid  proposal.  Any  repair  manual  can  become 
very  easy  to  read  if  it  is  allowed  to  contain  little  or  no 
substance . 

The  FAR,  (based  on  OMB  Circular  A-76,  chapters  5  and  6) 
specifies  that  the  government  must  recognize  the  differences 
between  the  military  and  the  commercial  worlds.  The  US 
Government  must 


(a)  rely  generally  an  commercial  sources  for 
supplies  and  services,  if  certain  criteria 
are  met,  while  recognizing  that  some 
functions  are  inherently  Governmental  and 
must  be  performed  by  Government  personnel, 
and  <b)  give  appropriate  consideration  to 
relative  cost  in  deciding  between  government 
and  contractor  performance.  <32:7-4.1) 


DoD  Directive  4151. 1,  Use  of  Contractor  and  DoD 
Resources  for  Maintenance  of  Material  supports  this, 
stipulating  that 

prime  consideration  shall  be  given  to  the  use 
of  contractor  support  for  indirect  (Depot) 
maintenance  . . . C and]  Depot  capacity. .. shal 1 
be  kept  to  the  minimum  required  to  ensure  a 
ready,  controlled  source  of  competence  and 
resources  necessary  to  meet  war.  <35:E8,  E9> 


Army  regulations  also  support  this.  AR  750-2  Army 
Maintenance  Wholesale  Operations,  specifies  that  workload 
beyond  the  capability  of  a  Depot  may  be  contracted  to 
commercial  sources,  but  may  not  exceed  20%  of  the  total 
dollar  value  of  the  authorized  in-house  Job  order.  (27:5-4) 
Additionally,  The  Department  of  Defense  has  directed  that 
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each  DoD  component’s  organic  Depot 
maintenance  peacetime  physical  capacity  shall 
be  planned  to  accomplish  no  more  than  70 
percent  of  its  gross  mission-essential  Depot 
maintenance  workload  requirements.  At  least 
30  percent ... shal 1  be  decided  on  the  basis  of 
economy,  the  timely  availability  of  private, 
commercial  sources,  and  the  need  to  maintain 
a  commercial  industrial  mobilization  basis. 
<35: E9c> 


These  last  four  references  are  of  particular  relevance 
to  the  research  topic  because  they  require  the  military  to 
fight  becoming  a  complacent  <and  inefficient)  monopoly,  and 
require  that  military  specifications  be  honored  if  possible. 
Also,  these  references  caution  that  cost  <and,  by  extension, 
manual  use)  comparisons  must  be  done  with  care  and 
consideration  of  industry  differences.  As  reported  in  the 
TRW/Air  Force  study  discussed  above,  some  specifications  in 
Technical  Order  manuals  and  Commercial  manuals  have  been 
industry  specific,  and  should  have  been  allowed  to  exist. 
This  finding  is  applicable  to  the  present  as  well. 

Both  the  military  and  commercial  sectors  have  been  found 

to  have  a  similar  reliance  on  an  engineering  focus  for  their 

manuals,  rather  than  a  user  focus.  (46:385)  This  could 

contribute  to  ’’difficulty  of  use.”  However,  one  researcher 

still  feels  that  the  military  has  made  more  progress  towards 

producing  effective  texts  than  the  commercial  sector  has. 

<46:385)  The  military  has 

contributed  to  the  development  of 
specifications  for  effective  designs. . . 
provided  guidance  in  explicit  detail... 
achieved  consistency  of  design. ..and  provided 
detailed  rationale  for  their  designs 
(ineffective  as  some  of  them  £  have  been]) 

The  commercial  world  has  not!  (46:385) 
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The  private  sector  has  made  some  relative  strides  beyond 
the  military  in  manual  readability  technology,  however.  Both 
Caterpillar  Corporation  and  McDonnell  Douglas  have  developed 
forms  of  ’’simplified  English”  which  they  have  used  for  some 
of  their  manuals.  Their  experience  is  that  simplified 
English  "was  hard  to  use  at  firsthand  productivity  dropped 
at  first,  but  soon  returned  to  normal.”  (47:25)  The 
relevance  of  this  information  to  the  research  topic  is  that 
it  shows  the  relatively  radical  procedures  the  commercial 
world  has  taken  to  cut  costs,  but  which  still  have  not 
especially  improved  manual  usability. 

The  reports  by  the  Logistics  Management  Institute  <LMI> 
compared  commercial  and  depot  modernization  tactics.  Though 
not  directly  related  to  "the  usability  of  DMVR, ”  these 
reports  represent  a  body  of  data  collected  on  management 
strategy  differences  in  the  two  maintenance  sectors.  These 
data  are  important  in  order  to  know  management’s 
modernization  philosophies,  which  could  explain  differences 
in  the  relative  work  done  to  make  manuals  usable. 

The  first  LMI  report  surveyed  (5:12)  outlined  the  scope, 
cost  and  complexity  of  modern  DoD  Maintenance  depots  with 
emphasis  on  the  ways  they  have  modernized  operations  to 
Increase  cost  effectiveness.  The  report  showed  that  ’’the 
same  things  which  have  closed  private  corporat ions-- 
management  complacency,  poor  business  practices,  excessive 
redundancies — exist  in  Depots.”  (5:12) 

Helicopter  component  repair  cycle  times,  shown  to  be 
’’much  too  lengthy,”  (5:11)  were  cited  as  a  result.  The 
report  did  not  consider  that  maintenance  manuals  have  been 
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shown  to  have  an  effect  (perhaps  an  even  more  direct  effect) 
on  repair  cycle  times  than  has  management.  <19:60) 

Another  LMI  report  (11)  discussed  modernization 
strategies  of  four  Depots  and  from  three  major  corporations 
of  the  private  sector.  It  showed  that  v.'hen  either  the  DoD 
or  the  private  sector  focused  first  on  simplifying 
’’processes’*  and  later  introduced  automated  production  and 
inventory  controls,  modernization  was  better  realized. 
(11:111)  Conversely,  ’’several  companies  even  said  that  they 
made  costly  mistakes  by  prematurely  applying  high  technology 
to  complex,  poorly  understood  processes.”  <11:4-1)  In  other 
words,  addressing  the  problem  of  high  cost  Depot  repair  at 
the  operational  process  or  causal  level  of  the  problem  is 
the  best  approach.  "Operational  processes”  of  maintenance 
are,  for  instance,  manuals  that  are  readable,  have  many 
illustrations,  and  have  published  ’’aids”  that  promote 
understanding. 

The  process/product  theory  applies  nlso  for  —  r.ual 
writing.  Significant  research  has  shown  that  the  product 
oriented  approach  to  writing  (using  specifications)  is  "an 
inadequate  means  of  controlling  the  product.”  (45:76) 

Rather,  research  suggests  that  "the  focus  must  be  on  the 
expertise  of  the  writer  and  on  the  process  of  writing” 

(45:76)  to  get  usable  manuals.  The  manual  itself  should  not 
be  written  unless  the  writer  is  uniquely  capable. 

The  next  LMI  report  surveyed  the  ’’problems  and  issues 
that  are  unique  to  contractor  repair.”  <15:c-l>  The  report 
selected  six  leading  firms  with  excellent  repair  management 
programs  which  also  repaired  items  comparable  to  those  of 
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DoD.  It  found  that  the  commercial  sector  had  ’’known  shorter 


repair  turnaround  times”  <15:C-2>  than  did  the  Depots. 
’’Turnaround  time”  is  the  time  needed  to  repair  a  piece  of 
equipment,  and  equates  roughly  to  customer  satisfaction.  The 
study  said  that  one  of  the  causes  of  the  difference  was  that 
companies  visited  made  a  ’’conscious  commitment  to  minimize 
repair  cycle  times...  at  the  expense  of  repair  efficiency.” 
<15: D— 11)  Repair  efficiency  can  be  a  product  of  manuals 
designed  for  easy  use. 

Though  contractor  repair  turnaround  time  was  positive, 
other  elements  of  contractor  repair  were  negative  for 
producing  the  finest  examples  of  usable  manuals.  The 
contractor  repair  facilities  surveyed  were  characterized  by 

diminished  control  and  visibi lity . . . t and  had 
problems  in]  workload  forecasting  and 
scheduling,  contracting  procedures,  and 
performance  measurement  and  visibility. 

< 15 : C-l ) 

These  are  the  same  difficulties  that  the  Army  has  sought 
to  overcome  through  the  use  of  readability  tests  and 
detailed  manual  specifications.  <22: l-7>  However,  the 
commercial  world  has  not  highly  valued  these  aids  to  manual 
development.  For  the  commercial  repair  facility,  ’’the 
bottom  line  t is]  profit,  and  usability  I  is]  a  cost  burden 
and  [is],  at  best,  only  weakly  related  to  sales.”  <46:386) 
This  study  found  that  the  appreciation  of  manual  usability 
was  generally  lacking  in  the  commercial  world. 

The  relevance  of  the  foregoing  literature  to  the 
research  topic  is  that  it  presents  some  of  the  basic, 
work-related  differences  between  organic  military  Depots  and 
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commercial  repair  facilities.  These  differences  should  be 
taken  into  account  when  Judging  the  usability  of  a 
maintenance  manual  in  either  sector. 

Previous  Evaluations  of  CoTnirercial  Manuals  in  the  Depot 

Chapter  6  of  AR  25-30  specifies  when  and  under  what 
conditions  commercial  equipment  repair  manuals  may  be  used 
in  the  military.  It  does  not  specify  that  either  manual 
must  be  chosen  based  on  ease  of  use  or  updating,  however.  It 
specifies  that  commercial  manuals  may  be  used  when  they 
cover  the  majority  of  Army  requirements,  are  verified  by 
appropriate  testing  conducted  prior  to  the  procurement  of 
equipment,  and  are  for  off-the-shelf  commercial  items. 

(22:6) 

The  MRSA  DMVR  study  (previously  addressed)  found  that 
commercial  manuals  have  been  deficient  in  data  needed  for 
repair /overhaul  of  Army  equipment  and  have  not  addressed 
repair/overhaul  in  a  military  environment.  (8:2-69)  Test 
information  was  found  to  be  lacking,  especially  for 
electronic  equipment;  commercial  parts  lists  were 
incompatible  with  Army  parts  list;  many  needed  commercial 
design  illustrations,  being  proprietary,  were  not  shown  for 
patent  reasons;  and  commercial  mechanical  tolerances  far 
exceeded  those  allowable  for  Army  equipment.  (8:2-70)  All 
these  commercial  manual  attributes  added  to  the  amount  of 
supplementation  required  for  military  use,  which,  the  MRSA 
study  implied,  have  contributed  to  relative  difficulty  of 
use. 
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It  should  not  be  surprising  to  find  that  commercial 
manuals  could  be  harder  to  use  than  military  manuals.  The 
military  has  been  using  lesser  educated  technicians  for  a 
long  time — probably  longer  than  "the  rest  of  our 
technological  society,”  <45:70)  the  Carnegie  Communications 
Design  Center  pointed  out.  Lesser  educated  military 
technicians  have  needed,  and  have  been  given,  manuals 
designed  to  their  education  levels.  On  the  other  hand, 
commercial  mechanics  are  selected  because  they  are  experts, 
and  can  understand  more  complicated  procedures.  <45:70] 

An  Air  Force  Institute  of  Technology  study  determined 
the  adequacy  of  commercial  manuals  for  Air  Force 
intermediate  and  Depot  level  maintenance,  in  1989.  The  study 
conducted  an  opinion  survey  among  Air  Force  commercial 
manual  users  in  the  Depots.  Sixty  percent  of  the 
respondents  felt  that  commercial  manuals  were  inadequate. 
<6:46) 

However,  the  commercial  manuals  used  in  the  Depots  which 
were  determined  to  be  adequate  for  use  by  the  user  actually 
had  not  been  thoroughly  reviewed.  <6:viii>  Manuals  found  to 
be  inadequate  were  reviewed  by  personnel  most  thoroughly 
qualified  to  review  manuals  <e.g.,  the  technical  order 
specialist,  the  equipment  specialist,  the  maintenance 
technician,  and  the  using  command).  The  manuals  found  to  be 
’’adequate"  were  reviewed  by  "personnel  having  the  least 
knowledge  on  the  day-to-day  functions  of  technical  manual 
development."  <6:47)  The  reason  that  the  commercial  manuals 
were  found  to  be  inadequate  was  that  they  generally  lacked 
comprehensive  maintenance  and  safety  procedures,  and  did  not 
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display  the  needed  information  on  "hardware  accuracy,  data 
rights,  and  part  numbers."  <6:47>  In  any  event,  the  study 
found  that  the  commercial  manuals  used  were  generally  not 
adequate.  Ho  attempt  was  made  to  compare  Depot  vs 
commercial  manuals  for  readability  and  usability,  but  the 
study  recommended  that  a  follow-on  study  be  made  to  do  this. 
(6:53) 

The  MRSA  study  (previously  addressed)  discovered  that 

the  Army  Depots  have  accepted  commercial  manuals  with 

apparently  great  shortcomings.  It  found  that 

generally,  if  the  commercial  manual  is 
adequate  in  perhaps  half  or  2/3  of  the 
information  needed,  it  may  be  Judged 
acceptable  by  the  proponent  and  user. 

(8:2-42) 

The  relevance  of  the  previous  two  studies  to  the 
research  topic  was  that  they  showed  that  in  both  of  the 
existing,  available  studies  where  commercial  manuals  were 
compared  to  Depot  manuals,  the  usability  of  the  commercial 
manuals  was  seen  as  suspect.  However,  though  neither  study 
showed  that  commercial  manuals  were  especially  harder  to  use 
than  Depot  manuals,  both  showed  that  cown^rclal  manuals 
typically  have  less  information  in  them.  Further  research 
is  needed  to  determine  if  there  are  beneficial  usability 
aspects  of  commercial  manuals  that  could  be  incorporated 
into  military  manual  specifications. 

Readability  Analysis  in  Technical  Manuals 

There  are  many  types  of  evaluation  procedures  for 
textual  material.  Some  are  better  suited  to  certain  classes 
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of  publications.  However,  almost  any  evaluation  is  better 
than  none  to  assess  the  readability  of  written  material. 

<62: 418) 

The  choice  of  which  evaluation  or  combination  of 
evaluations  to  use  should  be  based  on  a  thorough  knowledge 
of  the  goal  of  the  verification  study.  If  the  results  would 
be  used  to  design  or  redesign  writing  systems,  many  varied 
tests  should  be  used.  If  the  goal  of  testing  would  be  a 
market  survey  of  existing  manuals,  fewer  or  less 
comprehensive  (and  less  expensive)  tests  might  achieve  the 
goal.  <62:419)  This  research  study,  therefore,  would  fall 
into  the  latter  category. 

Research  has  shown  that  there  is  "no  universally  correct 
way  of  presenting  information.’*  <62:425)  Information  is 
communicated  to  different  people  under  different 
circumstances,  through  multiple  channels.  (62:432) 

Therefore,  it  follows  that  individual  evaluation  techniques 
cannot  evaluate  every  possible  aspect  of  a  manual.  Such  is 
the  case  with  this  study.  The  purpose  of  this  research 
study  is  to  determine  if  the  DMVR  is  harder  to  use  than 
commercial  manuals,  so  the  goal  would  be  to  find  a  suitable 
number  of  verifiable,  comparable  characteristics,  and  simply 
present  the  set  with  the  larger  number  of  characteristics. 

For  the  purpose  of  this  research,  evaluation  testing  is 
indispensable,  but  otherwise,  most  publishers  are  little 
motivated  to  use  such  testing.  One  of  the  best  established 
materials  evaluation  theories  is  that  of  Learner 
Verification  and  Revision  <LVR) ,  which  has  been  promoted  for 
the  last  decade.  It  is  an  editorial  philosophy  of  verifying 
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an  individual  text’s  ability  to  communicate  effectively  with 
individuals,  based  on  feedback  from  learners.  (50:397) 
Although  LVR  has  been  lauded  by  academicians,  the  initial 
enthusiasm  for  LVR  ’’soon  dissipated  in  the  face  of...  apathy 
and  publisher's  resistance.”  (50:410) 

Many  researchers  have  denounced  most  readability 
formulae.  A  complete  description  of  their  criticisms,  and 
why  they  do  not  apply  here,  follows  in  the  next  section. 

AR  25-30  requires  publications  to  be  edited  to  make  them 
"more  usable,  understandable,  and  readable  in  accordance 
with  the  Army  Readability  Program.”  (22: l-20b)  However,  one 
study  of  four  commercial  publishers  of  military  manuals 
showed  that  none  had  planning  goals  that  took  the  reader 
into  account,  unless  the  military  specification  required  it. 
If  there  was  a  requirement,  it  was  translated  into  a  maximum 
sentence  length  requirement.  (45:76) 

notwithstanding  this  apparent  commercial  apathy  about 
manual  usability  testing,  over  100  user-oriented  design 
techniques  have  been  used  in  the  military  since  1950. 

(45:70)  Experimental  evaluation  of  these  techniques  has 
indicated  that,  when  properly  implemented,  they  yield 
significantly  improved  manual  user  performance.  (45:70) 
However,  one  study  showed  that  the  military  seldom  ever 
conducted  true  verifications  of  their  manuals  during  or 
after  manual  development,  nor  did  it  often  consider  audience 
and  task  context  during  document  development.  (45:75)  This 
was  true  even  when  developers  knew  that  consistent 
coordination  between  the  equipment  developer/Depot  and  the 
manual  writer  could  make  a  manual  more  usable.  (45:75) 
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The  point  of  user  testing  is  that  the  best  manuals  are 
seen  as  clear  and  instructive  to  a  broad  spectrum  of  users. 
<45:74)  Inexperienced  (or  highly  experienced)  mechanics 
should  be  able  to  use  the  manuals  easily.  This  way,  the 
manuals  are  more  efficient,  as  well  as  easier  to  use  for 
either  set  of  mechanics. 

Vhen  evaluation  is  used,  a  conventional  method  is  some 
form  of  quest ion-and-answer  procedure,  (62:427)  where  a 
skilled  examiner  or  test  questions  a  reader  on  what  he  has 
read.  This  method  is  beet  for  text  with  a  strong  theme,  or 
much  symbology.  (62:427)  Maintenance  manuals  have  neither 
of  these  traits. 

For  maintenance  manuals,  a  good  but  expensive  method  of 
content  evaluation  is  "task  evaluation."  (62:428)  Through 
such  things  as  field  observations  or  laboratory  tests,  data 
are  gathered  on  how  well  a  task  can  be  performed  after  it 
has  been  described  in  a  text.  A  significant  drawback  of 
this  method  is  that  it  is  almost  impossible  to  be 
unobtrusive  enough  as  a  tester  to  be  able  to  get  unbiased 
results.  (45:75)  Therefore,  this  approach,  though  good  in 
theory,  normally  is  not  used  in  practice. 

One  of  the  most  frequently  used  methods  of  evaluation  is 
editorial  commenting.  (62:425)  This  method  is  objective, 
but  only  gives  slight  indication  of  possible  textual 
problems.  It  also  does  not  compare  existing  manuals  well, 
as  is  the  aim  of  this  research  study. 

Another  method  of  evaluation  is  called  "Cloze  testing," 
where  random  words  of  a  written  paragraph  are  erased,  and 
test  subjects  write  in  the  omitted  words.  (62:426)  It  is  an 
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inexpensive,  objective  test.  This  method  Indicates  how  the 
structure  of  a  paragraph  and  its  sentences  contribute  to 
overall  meaning.  It  is  most  appropriate  in  text  consisting 
of  paragraphs.  (62:426)  Maintenance  manuals  do  not 
generally  have  long,  thematic  paragraphs,  however,  which 
restricts  much  use  of  this  test  for  maintenance  manuals. 

Wright  advocates  an  evaluation  termed  "heuristic." 
(62:422)  The  heuristic  asks  an  evaluator  to  judge  text 
based  on  its  supposed  power  to  motivate  reading.  The  more 
it  motivates,  the  better  it  is.  Demotivators  in  "Component 
psychological  processes”  (e.g.,  pattern  recognition,  memory, 
and  comprehension)  make  a  manual  less  useable.  (62:423)  In 
a  maintenance  manual,  motivators  of  understanding  are  such 
things  as  understanding  assistance  cues,  and  illustrations. 

The  heuristic  approach  cannot  be  objectively  validated, 
but  its  theory  is  rooted  in  logic  as  well  as  in  much 
psychology  research.  (62:422)  Judging  a  text’s  usability 
based  on  the  heuristic  is  also  quick  and  easy.  As  long  as  a 
list  of  "demotivators”  is  known,  then  their  occurrence  in  a 
test  can  be  both  counted  and  compared  to  examples  found  in 
other  texts. 

"Demotivators"  of  textual  material  are  such  things  as 
the  poor  placement  of  illustrations  with  respect  to  text, 
and  paragraphs  of  monotonous  text  packed  together  tightly, 
without  helpful  cueing  devices.  To  perform  the  heuristic 
approach,  an  evaluator  counts  the  relative  occurrence  of 
interest  demotivators  between  two  manuals,  and  makes  a 
comparison. 

The  term  "heuristic"  has  a  negative  connotation,  because 
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one  common  definition  of  it  is  that  it  is  an  aid  to  problem 
solving,  but  otherwise  unjustifiable.  This  is  really  an 
unfortunate  designation,  because  in  some  applications,  such 
as  in  this  research  project,  the  heuristic  has  ample 
Justification.  A  more  accurate,  and  equally  used  definition 
of  ’’heuristic"  is  that  it  is  a  simple  aid,  perhaps  not 
exactly  precise,  but  nonetheless  serviceable.  Further 
discussion  on  the  justification  for  using  heuristic  tests 
for  the  purposes  of  this  research  study  follow  in  the  next 
section. 

Another  method  of  evaluation  discussed  in  the  literature 
is  that  of  readability  formulae.  Readability  tests  have 
existed  for  years.  A  complete  discussion  follows,  in  the 
next  main  subsection  of  this  chapter.  Readability  formulae 
are  well  suited  to  making  quick  objective  comparisons  of  two 
textual  passages.  <62:431) 

The  relevance  of  this  information  to  the  research  topic 
is  that  it  tends  to  support  the  fact  that  no  verification 
technique  is  fail-safe  or  is  sensitive  to  all  readers* 
abilities  or  problems  with  reading,  but  that  even  one  is  far 
better  than  none.  Certain  tests  appear  to  have  the  highest 
applicability  to  technical  manual  assessment,  are  typically 
objective,  and  lend  themselves  well  to  the  development  of  a 
relatively  quick,  inexpensive  body  of  data  for  comparing 
different  manuals.  The  best  tests  for  this  research  are 
Readability  formulae,  and  the  heuristic  approach  for 
counting  illustrations  and  cue  frequency,  and  the  user  test, 
as  limited. 
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The  Validity  of  the  Tests  to  be  Used 


Readability  In  texts  (Reading  Grade  Level)  aR  25-30 
discusses  the  methods  approved  for  use  in  measuring  the 
Readability  of  Army  publications.  The  “approved  method” 

<22: 1-28)  is  the  Kincaid  Readability  formula,  which  results 
in  a  usability  score  called  a  Reading  Grade  Level  <RGL) .  AR 
25-30  cautions  that  using  RGLs  to  measure  readability  is 
only  a  partial  indicator  of  readability  <22: 1-28) ,  but  it 
still  is  the  only  Army  approved  test  with  a  formula  which 
results  in  any  score  which  can  be  validated.  Therefore,  it 
is  the  formula  which  will  be  used  in  this  research  study. 

Studies  have  shown  some  shortcomings  in  readability 
formulae.  <62:423)  One  study  has  shown  that  revising  a  test 
to  meet  readability  criteria  is  "no  guarantee  of  [getting 
the  result  of]  successful  communication.  <62:423)  By  this 
the  writer  means  that,  though  the  text  can  be  thus 
quantitatively  graded,  that  is  where  its  utility  ends.  A 
host  of  other  factors  internal  and  external  to  the  text  can 
still  block  learning  after  the  text  is  evaluated.  Another 
complaint  about  readability  formula  is  that  such  tests  are 
extremely  simplistic,  and  do  not  taking  into  account  the 
intended  use  of  the  manual.  <62:423) 

Redish  <56:46)  wrote  scathing  criticisms  in  1985  of  the 
use  of  readability  formulae  as  the  only  means  of  text 
evaluation.  However,  the  criticisms  apply,  in  every  case, 
to  subjects  outside  the  scope  of  this  research  study. 

Redleh  supported  the  use  of  RGL  tests  for  “comparison, 
objectivity,”  <56:46),  and  correlating  certain  features 
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between  texts.  (56:49)  Redlsh  only  supported  expensive, 
difficult  user  testing,  "to  test  everything. . .to  make  the 
document  understandable  and  useful.”  (56:51)  One  of  her 
main  arguments  was  that  RGL  testing  was  invalid  because  it 
did  not  test  for  important  comprehension  aids,  such  as 
"graphics  and  typography.”  (56:50)  RGL  tests  were 
designed  for  text  comparisons,  and  that  is  still  the 
approach  of  this  research  study. 

AR  25-30  stipulates  that  RGLs  of  target  audiences  will  be 
the  criteria  by  which  all  new  manuals  will  be  written,  but 
that  an  existing  publication  will  not  be  revised  solely  to 
lower  its  RGL.  (22: 2-10 (1)- (3) )  AR  25-30  also  says  that  for 
equipment  repair  manuals,  the  RGL  will  be  for  the  target 
audience,  but  not  to  exceed  RGL  12.  (22:2-10b2>  And, 

equipment  developers  will  determine  that  manufacturers’ 
equipment  publications  are  acceptable  for  use  only  when  they 
meet  the  requirements  of  XIL-M-7298  (not  on  the  basis  of 
RGLs).  (22:2-1062) 

The  Kincaid  readability  Formula  measures  the  average 
words  per  sentence  of  a  text,  compares  it  to  the  average 
syllables  per  word,  and  indicates  a  reading  Grade  Level 
(RGL)  for  text  tested  (see  figure  Al).  The  RGL  is  a  scale 
from  5  to  34,  with  each  number  roughly  equal  to  a  grade 
level  in  the  American  school  system,  until  grade  17  (College 
level)).  Numbers  above  level  17  equates  to  simple  ’’reading 
difficulty  levels."  (22: fig  2-1). 

The  original  Flesch  Readability  Formula  included 
another,  similar  test,  called  the  "human  interest  level,” 

( 1 : front ispiece )  which  compared  the  percent  of  "personal 
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words”  to  the  percent  of  "personal  sentences”  In  a  text, 
resulting  In  a  score  on  "dullness"  (see  figure  A2).  Flesch 
felt  that  the  more  "personal”  words  In  a  text,  the  more 
interesting  the  text  would  be.  (1:251)  The  Army  rejected 
this  part  of  Flesch’ s  formula,  though  current  research  has 
shown  that  personal  words  in  a  text  increase  interest  paid 
to  printed  material.  (60:184) 

The  emphasis  placed  on  RGLs  by  AR  25-30  appears  to  be 
determined,  yet  tentative.  That  is,  this  test  has  been 
authorized,  but  its  wholesale  use  has  been  constrained  for 
use  only  under  certain  circumstances.  This  appears  to  lend 
precautionary  notice,  (added  to  the  previous  precaution 
about  this  method) ,  which  should  be  remembered  during  the 
evaluation  of  data  gathered. 

Heuristic  (Illustrations  in  texts).  Illustrations  are 
important  to  DJCWRs,  because  illustrations  aid  understanding 
in  many  ways.  First  and  foremost,  illustrations  "can  help 
the  reader  get  the  facts  and  ideas  in  the  text.”  (2:182) 

They  help  the  reader  learn  the  fact  or  idea,  because  it  has 
been  found  that  "it  is  easier  to  tie  a  memory  to  a  picture 
than  an  idea.”  (3:91)  Illustrations  are  also  essential  to 
"amplify  and  stress  a  central  theme  or  idea  already 
expressed  in  the  text.”  (52:739)  Illustrations  describe 
complex  objects.  In  fact,  a  direct  relationship  has  been 
found  between  the  necessity  of  having  illustrations  in  a 
text,  and  the  complexity  of  the  material  to  be  presented. 
(52:73°)  In  general,  the  more  illustrations,  the  better. 

Dwyer  points  out  that  in  over  30  independent  studies  and 
100  experiments  conducted,  color  has  been  found  to  be  "a 
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viable  instructional  variable”  <0a: 149)  which  can  improve 
illustrations.  Dwyer  also  points  out  research  in  learning 
theory  that  shows  that  learning  is  more  complete  and 
facilitated  when  there  is  more  realism  in  the  training 
<0:4).  He  shows  that  the  simple  black  and  white  line 
drawing,  of  8  possible  illustration  mediums,  ranks  last  in 
realism  and  efficiency  in  facilitating  learning  <0:5). 
However,  MIL-M-38784B.  the  military  general  style 
specification,  stipulates  that  military  illustrations  will 
be  used  "in  lieu  of  photographs.”  (32a: 3. 6.2) 

Heuristic  (Cuing  Aids  in  Texts).  Much  has  been  written 
in  psychology  journals  about  the  learning  process,  and 
specifically,  about  the  need  for  ’’management  of  the  reader’s 
attention  tin  reading].”  (44:192)  Management  aids  such  as 
"typographical  cues”  help  readers  understand  written 
material.  (44:192) 

Understanding  is  a  complex  activity  which  involves  the 
simultaneous  process  of  recognizing,  identifying, 
organizing,  and  Integrating  ideas,  text,  and  meanings. 

(44: 193)  All  these  actions  compete  for  a  limited  amount  of 
human  memory  space.  (44: 194)  Readers  cope  with  this 
situation  by  ’’allocating  their  attention  differentially  to 
text  information."  (44:194) 

Readers  define  and  decode  the  organization  of  a  text  and 
internalize  the  information  sequentially,  that  is,  after 
progressing  through  the  text  and  encountering  the  author’s 
meaning  cues.  (44:195)  If  the  author  makes  these  cues  stand 
out,  by  underlining  them,  pointing  to  them  with  arrows,  or 
summarizing  them  by  headings,  it  helps  the  reader’s 
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"understanding  workload.”  <44:194) 

Research  has  shown  that  when  text  has  no  published 
"cueing  aids,"  readers  provide  their  own.  In  a  survey  of 
200  randomly  selected,  used  college  textbooks,  92%  were 
found  to  contain  cues  written  in  by  students.  <44: 195) 

Cues  were  also  found  to  aid  readers  interpreting  complex 
tables,  graphs,  and  illustrations  <44:199)  such  as  are  found 
in  maintenance  manuals.  These  cues  "focus  the  readers 
attention  on  the  critical  components. .. to  break  the  figure 
into  its  component  parts  and  simplify  its  interpretation." 
<44: 199) 

In  his  studies  of  the  "Physiological  Effects  of  Cueing,” 
Dwyer  discusses  the  foundation  principle  supporting  the 
modern  typesetting  practice  of  double  column  paragraphing 
<0a:153).  As  the  human  eye  can  only  sharply  focus  in  a 
"small  foveal  region  of  about  2  degrees,  and  the  rate  at 
which  this  narrow  beam  of  sharp  vision  is  limited  to  about 
3-5  degrees/second"  <0a:153),  the  narrower  the  column,  the 
quicker  and  more  efficient  reading  is.  MIL-M-38784B  also 
requires  that  "unless  otherwise  specified,  manuals  shall  be 
prepared  double  column."  <32a:3.2.2)  However,  of  the  30 
technical  manuals  annotated  in  the  bibliography  of  this 
study,  including  new  and  old  manuals,  only  19%  of  the  DMVRs 
were  double  column,  but  67%  of  the  commercial  manuals  were 
double  column.  The  range  of  dates  for  the  DMVR  manuals  in 
double  column  format  is  from  1972-1986.  The  range  of  dates 
for  commercial  manuals  in  double  column  format  is  even 
greater,  from  1973  to  1988. 

User  Testing  and  User  Perceptions.  Much  has  been 
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written  about  mot ivators/demot ivators  to  reading  <44: 192, 
60:184,  0 : throughout ,  1 : throughout > .  When  a  reader 

perceives  that  a  text  is  easy  to  read  and  helpful,  he  tries 
hard  to  use  it  (and  the  opposite  is  also  true). 

A  reader  can  spend  much  or  little  time  reading  a 
technical  manual,  and  this  attention  can  be  measured.  More 
attention  is  given  to  good  manuals  than  to  bad.  User  testing 
<56  and  46)  tests  a  manual  by  observing  the  care  and 
attention  given  to  it  by  readers. 

There  is  one  aspect  of  user  testing  employed  in  this 
study,  namely,  gauging  the  interest  effect  of  RGL, 
illustrations,  and  cues  on  persons  who  are  inexperienced 
with  the  manuals.  Do  these  things  motivate  or  demotivate 
reading?  This  form  of  user  testing  agrees  with  other 
descriptions  in  Redish  <56)  and  Duffy  <4<i>,  involving  an 
inexperienced  user,  making  observations  about  a  manual’s 
technical  usability.  However,  in  this  study,  user  testing 
does  not  test  techniques,  but  is  used  as  a  vehicle  for 
recording  user  impressions  and  reading  motivation. 

User  testing  is  very  well  thought  of  by  researchers, 
because  it  tests  the  overall  quality  of  a  manual.  However, 
since  user  testing  is  so  difficult,  costly,  and  time 
consuming  <56:51),  this  study  uses  a  modified  test.  This 
test’s  biggest  divergence  from  ether  user  testing  (described 
in  the  literature)  is  in  its  method  chosen  to  record 
observations.  The  literature  describes  the  use  of  an 
independent,  trained  tester,  who  records  all  apparent . 
Important  observations  of  many  test  takers,  using  one  test, 
testing  one  manual.  (46)  In  this  study,  the  "user  test”  is 


-43- 


a  self  administered  test,  using  a  one-person  sample  (the 
researcher),  testing  many  manuals,  with  many  tests,  and  is  a 
careful  record  of  all  important  observations  from  the 
reader. 

Under  these  conditions  some  bias  is  possible,  because 
any  control  elements  of  the  test  amount  to  the  self  control 
of  the  researcher.  Therefore,  care  is  taken  by  the 
researcher  to  make  as  many  observations  as  possible,  without 
prizing  whether  those  recordings  represent  positive  or 
negative  observations.  Results  from  this  test  (when 
administered  carefully)  increase  the  chance  that  true 
interpretations  are  drawn  from  data  gathered,  because  its 
qualitative  data  couples  with  the  quantitative  data  of  other 
tests,  and  helps  verify  the  data. 

The  preceding  discussion  of  readability,  cuing  aids, 
illustrations  in  texts,  and  user  perceptions  is  important  to 
this  research  study  because  each  aspect  can  help  or  hinder  a 
mechanic  in  his  job.  An  overhaul  manual  with  these  aids  can 
help  the  average  mechanic  understand  his  work  well.  Such 
understanding  can  mean  fewer  work  errors  and  quicker  work. 

Summary 

This  chapter  contains  summaries  of  some  of  the  more 
significant  literature  concerning  the  topic,  ’’the  ease  of 
use  of  the  DXVR.”  It  discusses  the  many  differences  between 
the  commercial  and  Depot  scenarios,  with  implications 
concerning  manual  formats  and  Judgement  of  the  usability  of 
the  DM VR.  It  shows  previous  comparisons  of  Depot  and 
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commercial  manuals,  and  shows  that  relatively  few  real 
usabl 1 ity  differences  have  been  researched/discovered  so 
far.  Many  simple  methods  to  do  this  study  exist,  and  were 
discussed,  with  the  aim  of  showing  their  applicability  to 
evaluating  maintenance  manuals.  Some  research  findings  were 
presented  on  the  importance  of  instructional  cues, 
illustrations,  readability  in  texts,  and  user  perceptions, 
in  order  to  further  support  a  testing  procedure  for  this 
study.  Three  tests,  readability  testing,  heuristic  tests, 
and  user  tests  (including  text  "characteristics'*  and  point 
size  and  page  layout  "observations")  seem  to  be  best  suited 
for  this  research  and  therefore  are  the  tests  used. 
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CHAPTER  III 


RESEARCH  METHODS 


The  Data 

The  data  of  this  research  were  of  two  kinds:  primary 
data  and  secondary  data.  The  nature  of  each  of  these  two 
types  of  data  is  given  below. 

The  Primary  Data.  The  Depot  Maintenance  Work 
Requirements  and  the  commercial  maintenance  manuals  in  each 
commodity  area  were  the  sources  of  primary  data.  The  data 
were  quantitative,  such  as  the  number  of  occurrences  of  an 
aspect  of  usability  (i.e.  illustrations  or  cues),  an  RGL 
score,  or  a  format  style,  such  as  the  use  of  double  column 
paragraphs  or  the  letter  point  size  of  the  two  sets.  The 
data  were  also  qualitative,  such  as  the  characteristic 
style,  form,  or  content  which  could  effect  a  reader’s 
perceptions  about  readability  or  understanding.  (56:51) 
Hereafter,  the  word  ’’characteristics”  relates  to  the  user 
tests,  and  the  word  "observations”  relates  to  the  findings 
for  point  size  and  page  layout  (double  column  paragraphs) . 

The  Secondary  Data.  The  secondary  data  were  published 
studies,  reports,  texts,  and  articles,  and  the  unpublished 
user  letters  and  messages  dealing  with  specifications, 
contracting-out,  previous  attempts  at  manual  evaluation,  and 
usability  evaluation  theory.  The  data  were  qualitative, 
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representing  theories,  ideas  and  concepts. 


The  Criteria  for  admissibility  of  the  Data 

Only  published  DMWRs  and  published  commercial 
maintenance  manuals  for  diesel  truck  engines,  helicopter 
turbines,  and  radio  receivers/transmitters,  were  used  in 
this  study. 

The  Research  Methodology 

Wright  pointed  out  that  certain  tests  for  quantifying 
the  usability  and  readability  of  texts  could  be  used  better 
by  certain  researchers  and  applied  better  to  certain 
research  projects.  <62:427)  While  it  was  admitted  that 
’’almost  any  evaluation  is  better  than  none”,  (62:418)  the 
choice  of  assessment  procedure  would  have  to  take  into 
account  the  "certain  classes  of ...  communication  that  might 
be  crucial  for  that  text. ..in  order  to  attain  the  specific 
[research]  objective".  <62:418) 

As  the  aim  of  this  research  study  was  to  compare 
competitive  texts,  the  choice  of  techniques  for  this  study 
was  based  upon  selecting  those  tests  and  theories  that 
compared  texts.  The  tests  which  were  chosen  were  suited  to 
measuring  relative  quantities  of  common  attributes  of 
manuals,  e.g.,  short  paragraphs,  many  illustrations,  and 
extremely  specific  textual  passages. 

Redlsh  and  others  adamantly  championed  user  testing  as  a 
test  available  "to  test  everything  [well]’*,  <56:51)  and  an 
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attempt  was  made  in  this  research  effort  to  provide  some 
aspect  of  user  testing,  commensurate  with  the  study’s  scope. 
The  researcher,  representing  an  "inexperienced”  reader  of 
the  publications  (45:74),  recorded  many  characteristics  of 
the  manuals  which  seemed  to  be  either  easy  or  difficult  to 
read  or  understand,  or  which  could  be  either  easy  or 
difficult  for  another  reader.  These  ’’characteristics” 
(anything  that  makes  reading  productive,  tiresome,  fun,  or 
difficult,  etc)  have  been  shown  in  the  research  to 
positively  or  negatively  motivate  a  reader  to  read.  (62:422) 
The  remainder  of  this  study  will  use  the  terms  ’’positive” 
and  ’’negative”  reading  motivators  when  discussing  these 
characteristics. 

The  fact  that  these  data  were  collected  only  from  one 
observer  does  not  necessarily  mean  that  the  sample  size  was 
unacceptable,  because  the  data  came  from  a  large  sample  of 
manuals.  No  attempt  was  made  to  unnaturally  "manufacture” 
comments  to  record,  nor  was  any  attempt  made  to  balance  the 
record  of  positive  or  negative  motivators,  nor  was  any 
attempt  made  to  record  data  supportive  of  any  bias  of  the 
researcher.  However,  the  ’’characteristics”  were  required  to 
be  logical,  supported  by  research  known  at  the  time,  and 
substantive.  The  bulk  of  the  readability  characteristics 
from  the  user  tests  (see  Appendix  J)  were  based  on  a  set 
locus  of  research  questions. 

Leedy  defined  the  research  method  most  appropriate  for 
collecting  data  from  simple  physical  observations  as  the 
simple  survey,  and  specifically,  the  descriptive  survey. 
(4:88)  This  was  the  method  used  in  this  study.  Primary 
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data  sources  for  this  study  were  surveyed  with  tests  of 
usability,  e.g. ,  the  Kincaid  Reading  Grade  Level  <RGL> 
formula,  and  two  heuristic  counting  tests,  for  illustrations 
and  instructional  cues.  ’’Character ist ics”  (such  as  those 
that  might  be  made  in  a  user  test  by  any  average, 
inexperienced  reader)  and  ’’observations”  (those  of  either 
viewing  of  the  user  test,  of  point  size  data,  or  page  layout 
style)  were  collected  by  the  researcher  as  they  became 
evident  in  the  research  data  collecting  effort. 

Specific  Treatment  of  the  Data  for  Each  Subproblem 

Subproblem  one.  The  first  subproblem  was  to  determine 
the  relative  simplicity  of  the  manuals,  as  in  their  ease  of 
readability. 


The  Data  Needed 

The  data  .  eded  for  salving  subproblem  one  were  the  RGLs 
of  a  cross  section  of  selected  DXWRs  in  each  of  the  three 
equipment  categories,  the  RGLs  of  a  cross  section  of 
selected  commercial  manuals  in  each  of  the  three  equipment 
categories,  and  the  user  test  observations  and 
characteristics  from  each  of  the  manuals.  Five  manuals  each 
from  both  the  Depot  and  commercial  sector  for  each  of  the 
three  equipment  categories  were  compared,  so  that  a 
reasonably  representative  sample  size  was  studied  and 
therefore  any  significant  bias  was  reduced. 
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The  Location  of  The  Data 


DMVRs  were  located  in  Army  Material  Command/MaJ or 
Subordinate  Command  <AMC/MSC)  Publications  Centers  at  the 
following  addresses: 


1.  Commander,  USA  Aviation  Systems  Command  AVSCOM 
Attn:  AVSAV-MCTP 

4300  Goodfellow  Blvd 
St  Louis,  Mo  63120-1798 

2.  Commander,  USA  Tank  and  Automotive  Cmd  TACOK 
Attn:  AMSTA-MBC 

Warren,  MI  48397-5000 

3.  Commander,  USA  Communicat ions/Blectronics  CECOM 
Attn:  AMSEL-LC-ME-P 

Fort  Monmouth,  NJ  07703-5000 


Commercial  manuals  were  located  at  the  following 
commercial  business  addresses: 

1.  Superior  Diesel  Service 
2106  E  Main  Street 
Richmond,  Va 

phone  <804)  643-4021 

2.  Commonwealth  Jet  Services  Co 
Sandston,  Va 

phone  <804)  222-5474 

3.  Aero  Services 
Richmond  Airport 
Richmond,  Va 

phone  <804)  226-7231 

4.  Dominion  Communications  Systems 
135  Pickwick  Ave 

Colonial  Heights,  Va 
phone  <804)  526-6373 


The  Means  of  Obtaining  the  Data 
A  letter  was  sent  to  each  DMVR  manual  owner  requesting 
six  DMVRs,  by  designation  and  number.  These  were  sent, 
studied,  and  returned  <if  return  was  requested).  For 
commercial  manuals,  the  above  mentioned  distributors  were 
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visited,  and  the  manual  was  surveyed  in  the  distributor’s 
office.  Data  relating  to  each  of  the  six  tests  <RGLs, 
illustrations,  cues,  user  test  characteristics,  point  size 
observations,  and  page  layout  style  preferences)  were 
collected  from  the  manuals. 

The  Treatment  of  the  Data 

Only  the  two  manual  sets  in  the  three  categories  were 
used.  Individual  manuals  were  selected  from  these 
categories  by  a  random  choice  from  DA  PAM  25-30, 

Consolidated  Index  of  Army  Publications.  30  Jun  1989.  The 
choice  was  made  from  the  subheadings:  Engines,  Diesel; 
Engines,  Aviation;  and  Radio,  Receiver.  Six  selections  each 
were  made,  then  their  military  designations  were 
cross-referenced  to  DMVR  designations  from  AMC  PAM  310-9, 
Index  to  DMVRs  in  Print.  1989.  However,  when  the  requested 
manuals  were  not  available,  alternates  (judged  acceptable  by 
the  above  criteria)  were  accepted. 

How  the  Analysis  was  Made 

The  RGL  usability  test  was  performed  separately  on  each 
manual.  The  scores  were  entered  into  the  summary  table  (see 
Appendix  E) .  An  average  of  the  scores  was  determined  for 
each  commodity,  test  and  repair  sector.  These  were  also 
entered  into  the  summary  table. 

The  following  formula  was  used  to  calculate  the  RGL: 

Select  a  150  word  passage.  Begin  with  a 
paragraph  or  selection  and  continue  to  the 
end  of  a  complete  sentence,  even  though  it 
may  exceed  150  words 
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STEP  1 

Figure  the  average  sentence  length.  (Divide 
the  number  of  words  by  the  number  of 
sentences. ) 

<150  /  10  *  15) 

STEP  2 

Figure  the  average  number  of  syllables  in 
each  word.  Count  numbers  as  one  word.  Count 
acronyms  and  abbreviations  as  one  syllable 
unless  they  spell  a  word  of  more  than  one 
syllable.  Count  the  syllables.  Divide  the 
number  of  syllables  by  the  number  of  words 

<300  /  150  =  2.0) 

STEP  3 

Compute  the  RGL  on  the  nomograph  (see 
appendix  C) .  Connect  words  per  sentence  and 
syllables  per  word  using  a  straightedge.  The 
point  where  the  line  crosses  the  RGL  scale 
showed  the  reading  grade  level. 

(RGL  =  13.7)  <22: fig  2-1) 

The  following  rules  were  followed  for  the  counting  of 
RGLs: 


1.  Count  syllables  the  way  the  word  is  divided  in  the 
dictionary,  the  way  the  word  would  correctly  be  spoken. 

2.  Count  numbers  as  one  word. 

3.  Count  acronyms  and  abbreviations  as  one  syllable, 
unless  they  spell  a  word  of  more  than  one  syllable.  For 
instance,  TRADQC  (Training  and  Doctrine  Command)  constituted 
2  syllables,  CINCEUR  (Commander  in  Chief,  Europe) 
constituted  2  syllables,  and  SQT  (Skill  Qualification  Test) 
constituted  one  syllable.  (22:  fig  2-1) 

4.  Count  the  next  closest  suitable  page  if  an 
illustration  page  was  identified  to  be  used. 

5.  Count  each  non-blank  page  individually. 

6.  A  150  word  passage  could  be  an  entire  page  of  many 
small  paragraphs. 

7.  Select  a  starting  point  for  counting  at  random  on 
the  page. 
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8.  Count  titles/headings  that  come  within  the 
selection  body,  but  ignore  figures. 

9.  Count  hyphenated  words  as  two  words, *and  hyphenated 
words  or  numbers  as  one  syllable  either  side  of  the  hyphen. 

10.  Count  titles  as  one  sentence  if  they  contained  more 
than  one  word  or  if  they  headed  a  multi -paragraph  section. 

11.  A  "note"  or  "caution”  heading  belonged  to  the 
following  sentence,  and  was  therefore  not  counted  as  a  whole 
sentence,  even  if  it  preceded  a  note  or  caution  statement  of 
more  than  one  paragraph. 

12.  A  colon  or  semicolon  did  not  constitute  a  sentence 
break. 

13.  In  DMWRS,  count  titles  as  two  words  <DXVR  plus 
number),  even  though  the  title  contained  many  hyphens.  Count 
paragraph,  figure,  and  title  number  designations  <i.e., 

5-46)  as  one  word  and  one  syllable.  Count  National  Stock 
Numbers  <NSNs)  as  two  words  and  two  syllables  (NSN  plus  #> . 

14.  If  the  DMVR  was  too  small  to  have  14  sample  pages, 
but  mentioned  a  Technical  manual  <TM)  which  was  used  to 
amplify  the  DMVR,  then  count  the  TM  as  an  integral  part  of 
the  DMVR. 

15.  Count  misspelled  words  as  they  should  be,  not  as 
they  were  represented  in  the  text. 

16.  Do  not  count  RGLs  (or  pages,  cues,  or 

i 1 lustrat ions)  of  an  appendix,  index,  or  table  of  contents. 

Sample  Size 

To  select  a  manageable  sample  size,  and  to  facilitate  a 
sampling  variance  from  the  true  score  which  would  be  as 
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small  as  possible,  the  fallowing  procedure  was  selected.  An 
arbitrarily  chosen  range  of  allowable  percent  variances 
between  0  and  15  was  plotted  on  the  ’x  axis  of  a  graph  (see 
appendix  I).  An  arbitrarily  chosen  range  of  sample  sizes 
between  0  and  16  was  plotted  on  the  y  axis.  The  sample  size 
corresponding  to  the  intersection  of  the  5th  percent 
variance  level  and  a  derived  concave  frequency  distribution 
was  selected.  The  sample  size  of  14  was  derived  by 
averaging  10  random  RGL  samples  from  one  manual  as  a 
standard,  then  plotting  the  percent  difference  from  other 
sized  samples  against  it  on  the  graph.  RGL  samples  above 
size  14  produced  small  changes  in  x.  A  variance  of  5%  was 
selected,  and  this  is  also  a  commonly  accepted  allowable 
variance  <3a:507). 

The  following  procedure,  called  "percentile  page 
determination"  was  used  to  sample  each  manual  for  data  for 
the  RGL  only.  This  was  a  modified  version  of  the  sample 
selection  criteria  from  KIL-K-36784B,  changed  to  allow  for  a 
larger  sample  size  <32a:4.4.1>.  Characteristics, 
observations,  illustrations,  and  cues  were  not  collected 
based  on  the  percentile  page  method.  Their  occurrences  were 
not  uniform  enough,  throughout  the  manuals. 

The  percentile  page  procedure  allowed  equitable  sampling 
for  RGLs  from  each  section  nf  the  manual,  and  allowed  a 
large  sample  size.  The  sampling  procedure  also  kept  sample 
sizes  both  manageable,  and  of  equal  size  between  each 
manual.  This  was  very  important,  because  some  of  the 
manuals  varied  in  size  by  hundreds  of  pages. 

First,  the  number  of  pages  in  the  manual  was  determined. 


Then,  that  number  was  multiplied  by  .077  <7.7%)  so  as  to 
divide  the  manual  into  "percentile  pages”.  The  figure  .077 
was  derived  so  as  to  yield  14  equal  samples.  If  the  number 
computed  included  a  fraction,  the  score  was  rounded  to  the 
nearest  whole  integer.  Pages  corresponding  to  the  7.7th 
percentile  page  were  the  only  pages  studied.  By  this  method, 
only  14  pages  were  sampled  per  manual.  In  every  manual  the 
first  page  was  always  treated  by  the  tests. 

Characteristics  and  observations  were  counted,  based  on 
the  following  rules: 

1.  Scan  throughout  each  manual  once  for  general 
positive  or  negative  motivators,  and  record  each  impression, 
regardless  of  its  apparent  magnitude. 

2.  Record  characteristics  in  Chapter  IV,  under 
findings  relating  to  the  three  quant itative  tests.  Record 
observations  and  general  characteristics  in  separate 
sections  in  Chapter  IV. 

3.  Scan  no  more  than  30%  of  the  manuals  at  one 
sitting,  so  that  fatigue  does  not  influence  the  record. 

4.  Randomize  the  manuals,  and  alternate  between  Depot 
and  Commercial  manuals  during  the  scan,  to  further  reduce 
the  opportunity  for  user  (reader)  bias. 

5.  Scan  only  two  pages  of  each  manual  (one  with 
significant  verbage,  and  one  with  a  significant 
illustration)  and  test  each  for  any  significant  impressions 
relating  to  each  of  the  following  topics: 
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negative  motivators  positive  Tnntivators 

a.  Incomprehensible  aa.  Excellent 

sentence  meanings.  illustrations. 

b.  Light,  small,  or  bb.  Precise  words, 

blurry  lettering. 

cc.  Aura  of 

c.  Too  few  words  for  professionalism, 

completeness. 

dd.  Bold,  crisp 

d.  Too  many  words  to  lettering, 

be  considered  brief. 

ee .  Layout 

e.  "Busy”  (confusing)  motivates  respect, 

illustrations. 

ff.  information 

f.  Annoying  style  or  motivates  respect, 

form  of  any  kind. 

6.  Record  characteristics  obtained  from  paragraph  four 
in  appendix  J. 

Because  these  characteristics  were  essentially 
subjective,  weights  were  not  assigned  to  them.  Positive 
characteristics,  or  those  which  helped  understanding  or 
motivate  reading,  were  assigned  a  +1  score.  negative 
characteristics,  or  those  which  did  not  help  understanding  or 
motivate  reading,  were  assigned  a  -1  score.  All  scores  were 
added,  and  a  final  score  was  given.  Manual  final  scores 
(for  this  test)  were  converted  into  a  percentage  of  the 
total  of  the  characteristics  and  observations  made,  to  yield 
comparable  negative  and  positive  percentages. 


How  the  Data  Were  Interpreted 
Any  differences  in  RGL,  illustration  or  cue  scores  were 
tested  for  significance  by  using  the  Two-Way  Analysis  of 
Variance  (ANOVA)  test.  (3a: 503-508)  This  test  was  used 
because  it  compared  differences  between  manuals  within  a 
repair  category  as  well  as  between  Depot  and  commercial 
manuals. 
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The  independent  Two-Vay  ANOVA  statistic  showed  the  true 
relationship  between  the  variables  of  repair  sector  and 
equipment  category.  In  this  test,  the  mean  of  each  variable 
was  taken,  and  the  ANOVA  test  determined  the  statistical 
difference  between  the  two  means.  If  at  least  one 
significant  difference  was  shown,  its  related  hypothesis  was 
rejected.  This  was  repeated  for  each  of  three  tests. 
Characteristics  and  observations  user  tests  were  not 
verified  by  the  ANOVA  test. 

For  the  ANOVA  tests,  the  scores  for  each  of  the  five 
manuals  within  one  equipment  commodity  were  averaged,  and 
placed  in  the  ANOVA  table  (see  appendix  F).  Formulas  (see 
appendix  F)  were  applied,  and  the  results  were  placed  in  the 
ANOVA  summary  table  (see  Appendix  F) . 

The  summary  table  provided  two  calculated  scores  for  the 
ANOVA  test:  Ft,  comparing  the  treatment  of  commercial  and 
military  manuals  against  repair  scenarios,  and  Fb,  comparing 
the  three  commodity  blacks  against  each  other.  These  two 
statistics  were  compared  to  an  F  distribution  table  (see 
Appendix  H)  showing  numerator  and  denominator  degrees  of 
freedom,  and  a  .05  area  in  the  upper  tail,  F.  os.  A  larger 
Ft  or  Fb  value  than  the  table  F  value  would  cause  the 
hypotheses  to  be  rejected. 

Validity  of  the  Counts 

In  each  manual,  one  page  was  checked  for  counting  errors 
by  a  person  who  did  not  make  the  original  count.  If  that 
page  count  differed  from  the  original  count  by  more  than  2H, 
another  page  was  be  counted.  If  that  count  differed  by  more 
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than  2%,  then  all  counted  pages  in  that  manual  were 
recounted.  This  helped  ensure  correct  counts.  One  manual’s 
results  were  outside  the  2%  tolerance,  and  it  was  recounted. 

All  mathematical  computations  were  checked  by  being 
performed  a  second  time.  This  helped  ensure  accurate 
computations.  Each  time  information  was  transferred  from 
collection  sheets  to  the  draft  study,  the  figures  were 
double  checked  to  ensure  that  there  were  no  erroneous 
transferals. 

Subproblem  Two.  The  second  subproblem  was  to  determine 
the  relative  number  of  illustrations  per  manual  between 
manual  sets  (commercial  vs  DMVR) . 

The  Data  Needed 

The  data  for  solving  subproblem  two  were  illustrations 
in  each  manual,  converted  to  a  number  of  illustrations  per 
page  per  manual. 

The  Location  of  the  Data 

The  locations  of  the  manuals  were  the  same  as  for  the 
first  subproblem. 

The  Means  of  Obtaining  the  Data 

The  means  of  obtaining  the  data  were  the  same  as  for 
the  first  subproblem. 

The  Treatment  of  the  Data 

The  total  number  of  illustrations  in  each  manual  was 


counted  and  converted  into  a  percentage.  The  scores  were 
placed  in  the  summary  table  (appendix  E). 

The  definition  of  an  illustration  to  be  counted  as  a 
single  entity  was: 

1.  The  illustration  was  a  procedural,  descriptive 
graphic,  which  took  the  form  of  a  blown  up  picture,  a 
figure,  a  table,  or  a  list. 

2.  The  graphic  was  accompanied  by  its  own  sequential 
number,  recorded  below,  above,  or  beside  it. 

3.  The  graphic  was  referred  to  in  the  text. 

How  the  Data  were  Interpreted 

The  interpretation  of  data  was  the  same  as  for  the  first 
subproblem. 

Subproblem  Three.  The  third  subproblem  was  to  determine 
the  frequency  of  use  of  any  aids  of  understanding  in  the 
texts,  such  as  cues,  marginalia,  and/or  underlining. 

The  Data  Needed 

The  data  for  solving  subproblem  three  was  the  number  of 
cues  per  page  per  manual. 

The  Location  of  the  Data 

The  location  of  the  data  was  the  same  as  for  subproblem 

one . 


The  Means  of  Obtaining  the  Data 
The  means  of  obtaining  the  data  was  the  same  as  for 
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subproblem  one. 


The  Treatment  of  the  Data 

The  item  analysis  was  made  as  described  in  subproblem 
one.  The  following  rules  were  followed  in  determining  the 
constitution  of  a  cue: 

1.  A  cue  was  either  an  underline  of  a  title  or  an 
idea,  boldface  type  of  a  title  or  idea,  or  an  arrow  or  other 
typographic  accentuator,  used  for  the  purpose  of 
accentuation,  or  a  footnote. 

2.  A  cue  was  a  published  part  of  the  original  manual. 

3.  A  cue  was  reasonably  obvious  to  see  at  a  casual 
glance. 

4.  For  pages  written  only  in  boldface  or  with  all 
words  underlined,  only  cues  were  counted  that  were 
significantly  different  in  appearance  from  the  rest. 

5.  Multiple,  underlined  column  head  subtitles  on  one 
sentence  plane  (such  as  a  table  heading)  were  counted  as  one 
cue. 

6.  A  cue,  highlighted  to  make  it  more  visible, 
constituted  two  cues. 

7.  Page  numbers  were  not  cues,  but  page  headings  were 

cues . 

8.  Entire  paragraphs,  which  were  highlighted,  were 
counted  as  one  cue,  not  a  paragraph  of  cues. 

9.  Only  words,  not  acronyms,  were  counted  as  cues. 

How  the  Data  Were  Interpreted 
The  data  were  interpreted  as  stated  in  subproblem  one. 
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CHAPTER  IV 


RESULTS  OF  THE  STUDY 


Introduction 


This  research  study  compared  classes  of  equipment 
overhaul  manuals  of  the  US  Army  Depot  maintenance  system 
with  corresponding  classes  of  manuals  of  the  US  commercial 
equipment  maintenance  system  to  determine  if  Depot  overhaul 
manuals  were  more  difficult  to  use.  A  review  of  the 
significant  related  literature  indicated  that  such  a 
comparison  was  both  possible  and  needed:  ’’possible”  because 
other  studies  made  general  comparisons  of  the  two  manual 
sets  but  did  not  compare  usability,  and  ’’needed”  because  the 
proponent  for  Depot  manuals  (DESCOM)  has  no  published 
research  data  to  support  its  belief  that  Depot  Maintenance 
Work  Requirements  are  less  desirable  (harder  to  use)  than 
their  commercial  counterpart  manuals. 

Manual  Owner  Response.  Because  of  the  method  used  to 
sample  the  population  of  overhaul  manuals  from  the  Depot  and 
commercial  sectors,  100%  of  the  samples  requested  for  use  in 
this  research  study  were  obtained  and  studied.  That  is, 
thirty  manuals  were  requested  for  study,  and  thirty  manuals 
were  received.  Furthermore,  the  owners  of  the  manuals 
responded  quickly  and  accurately  to  provide  the  manuals  for 
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the  research.  In  a  few  cases  the  exact  manual  requested  was 
either  not  available  or  was  unsuitable  for  the  research 
(i.e.,  it  specialized  in  subjects  only  indirectly  related  to 
equipment  overhaul).  However,  replacement  manuals  were 
randomly  chosen  and  of  the  correct  specifications,  which 
fulfilled  the  selection  objectivity  requirement. 

The  Tests  Used.  The  related  literature  described 
verifiable  usability  tests  that  were  used  for  comparing  the 
two  sets  of  manuals:  the  (standard  US  Army)  Kincaid  Reading 
Grade  Level  (RGL)  test,  Heuristic  tests  to  compare  the 
frequency  of  use  of  illustrations  and  of  instructional  cues, 
and  a  modified  user  test.  The  results  of  all  but  the  last 
of  these  tests  were  compared  using  the  Two-Way  Analysis  of 
Variance  (ANOVA) ,  at  the  1st  and  4th  degrees  of  freedom,  to 
determine  the  degree  of  difference  between  the  two  manual 
sets.  The  last  test,  being  essentially  qualitative  in 
nature,  was  not  tested  with  the  Two-Way  ANOVA.  Since  this 
test  required  only  comparisons  of  simple  percentages 
made  of  the  differences  between  the  lists  of  ’’positive” 
and  "negative”  motivators,  no  statistical  test  was 
necessary. 

The  findings,  ANOVA  test  summary,  and  treatment  of  the 
hypothesis  for  each  of  the  tests  appear  below.  After  each 
"treatment”  and  "hypothesis"  appears  a  compilation  of 
usei — test  discovered  characteristics  related  to  that 
subject.  Each  test  of  the  research  methodology  produced 
findings  which,  when  interpreted  together,  provided  a 
plausible  reaponee  to  the  hypotheses  stated  in  this  study. 
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Reading  Grade  Level  <RGL) 


Findings.  The  first  hypothesis  of  the  study  stated 
that  there  was  no  significant  difference  between  the 
commercial  and  Depot  manuals’  readability,  as  measured  by  a 
comparison  of  their  reading  grade  levels.  Determining  the 
validity  of  this  hypothesis  required  comparison  of  Reading 
Grade  Level  <RGL)  data  collected  from  the  sample  population. 
The  results  are  summarized  in  Table  1,  which  compares  the 
average  overall  RGL  for  the  commercial  and  Depot  commodities 
of  truck  engines,  aircraft  engines,  and  radio  receivers/ 
transmitters.  The  individual  RGL  scores  were  averaged  from 
14  RGL  samples  in  each  text.  <See  Appendix  L  for  complete 
results. )  Smaller  values  are  better  than  larger  values. 

The  Table  shows  that  the  RGL  average  for  commercial 
truck  engines  was  1.9  grades  lower  than  the  Depot  average; 
for  commercial  aircraft  manuals  the  RGL  was  0.2  grades  lower 
than  the  Depot  average;  and  for  commercial  radio  receivers/ 
transmitters  the  RGL  was  1.1  grades  higher  than  the  Depot 
average . 
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Table  1 

RGL  DATA  SUMMARY 


Anova  test .  To  test  hypothesis  one,  the  ANOVA  test  was 
performed  to  measure  significant  differences.  <3a: 503-508) 
Table  2  summarizes  the  AITOVA  test  data.  The  Table  shows  the 
data  collected  from  the  Grand  Means  and  the  F  statistic 
calculations.  The  critical  value  in  the  Table  was  derived 
from  the  table  in  Appendix  H. 
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Table  2 

Two  Vay  ANOVA  Data  for  RGL 


Block 

MILITARY 

MANUAL 

COMMERCIAL 

MANUAL 

ROW  TOTALS 

ROW  MEANS  X 

Truck 

Engines 

(TE) 

9.3 

7.4 

16.7 

8.35 

Aircraft 

Engines 

(AE) 

9.2 

9.0 

18.2 

9.  1 

Radio 

Receivers 

(RR) 

9.2 

10. 3 

19.5 

9.  75 

Column 

Totals 

26.7 

27.5 

Column_ 
Means  X 

8.9 

9.2 

Grand  Mean  X  =  (X,  +  X2)  /  2  Grand  Mean  =  <XT6+  XA4+  X)  /  3 
(columns)  =  9.05  (rows)  =  9.07 


■ - 

Source  of 
Variation 

Sum  of  Squares 

Degrees  of 
Freedom 

Mean 

Sq  uare 

Test 

Statistic 

Treatments 
Military  & 
Commercial 

0.  135 

1 

0.  135 

=  0.  118 

Blocks 

TE.  AE.  RR 

1.962 

2 

0.981 

EB 

Error 

2.293 

2 

1.  147 

Total 

4.39 

5 

For  formulas,  see  Appendix  F. 

The  critical  value  from  Appendix  H,  with  1  and  4  Degrees  of 
Freedom,  equals  7.71. 
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Hypothesis.  With  1  and  4  degrees  of  freedom  at  the 
0.05  level  of  significance,  the  critical  value  of  7.71  was 
obtained  from  Appendix  H.  If  the  value  calculated  using  the 
Two-Way  AHOVA  test  was  less  than  or  equal  to  the  critical 
value,  the  hypothesis  should  be  accepted  (fail  to  reject) 

(3a: 507).  The  calculated  values  of  0.118  for  treatments  and 
0.855  for  blocks  were  both  much  lower  than  the  critical 
value.  Therefore,  the  stated  hypothesis  was  accepted.  There 
were  no  significant  differences  between  the  RGL  (i.e. 
readability)  of  commercial  overhaul  manuals  and  Depot 
overhaul  manuals. 

Related  Characteristics.  The  following  characteristics 
of  commercial  manuals  represent  the  opinion  of  a  manual  user 
(the  researcher)  recording  aspects  of  the  manuals  especially 
negative  (or  positive)  to  reader  motivation  (for  additional 
characteristics,  see  Appendix  J).  All  of  the 
characteristics  in  this  set  represent  the  initial  of  two 
viewings,  and  record  only  observations  relating  to  RGL.  (The 
first  three  "characteristics’'  are  negative  motivators  for 
commercial  manuals. ) 

1.  Detroit  manuals  had  long  paragraphs  and  much 
writing  per  page.  Less  writing  is  considered  better. 

2.  Mack  manuals  had  extremely  complex  instructions  and 
wordy  sentences. 

3.  Perkins  and  Pratt  Whitney  manual  letter  point  sizes 
were  so  small,  the  manuals  were  hard  to  read. 

The  following  characteristics  of  Depot  manuals  were 
especially  negative  to  reader  motivation: 
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1.  CECOM  DMWR  11-6625-2917—5  tad  many  typographical 
errors  in  it. 

2.  Most  DMVRs  were  laid  out  in  8  1/2  inch,  margin  to 
margin  sentence  lengths.  This  contributed  to  eye  strain. 
Only  19%  of  the  DMVRs  studied  used  double  column  style  (3 
of  16),  whereas  67%  of  the  commercial  manuals  studied  used 
double  column  style  <10  of  15). 

The  following  characterist ics  of  coTnunarc ial  manuals  were 

especially  positive  to  reader  motivation: 

1.  Caterpillar  manuals,  written  in  ’’Caterpillar 
Fundamental  English,”  had  very  little  unexplained  jargon. 

2.  Caterpillar  manuals  had  step  by  step  procedures 
which  were  extremely  easy  to  fallow. 

3.  Perkins  manuals  were  written  usually  in  one 
sentence  paragraphs.  This  tied  one  thought  to  one  repair 
step. 

The  following  observation  of  Depot  manuals  was 
especially  positive  to  reader  motivation: 

1.  The  point  size  for  the  letters  in  DMVR  words  was  in 
most  cases  larger  than  that  for  the  letters  in  commercial 
words.  Larger  letters  were  easier  to  read.  For  instance, 
the  average  point  size  for  DMVRs  in  this  study  was  7.4,  with 
a  size  range  of  7  to  8.  The  average  point  size  for 
commarcial  manuals  in  this  study  was  6.53,  with  a  size  range 
of  4  to  7.  There  was  an  11.8%  difference  between  the  point 
sizes  of  Depot  and  commercial  manual  letters. 
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Analyst?  for  Reading  Grade  Level  (RGL> 

Use  of  the  Two-Way  AlfOVA  statistical  test  on  the  RGL 
data  collected  from  DMVRs  and  comparable  commercial  overhaul 
manuals  revealed  that  there  was  no  statistical  difference 
between  the  two  manual  sets.  The  largest  difference  between 
any  set  of  manuals  occurred  between  commercial  and  Depot 
truck  engines  <1.9  RGLs).  The  smallest  difference  was 
between  commercial  and  Depot  aircraft  engines  <0.8  RGLs). 

The  greatest  variance  within  this  test  was  between  a 
commercial  truck  engine  manual  <RGL  5.5),  and  a  Depot  truck 
engine  manual  <RGL  10.1).  The  highest  and  lowest  RGLs  were, 
respectively,  a  commercial  truck  engine  manual  <RGL  5.5), 
and  a  commercial  radio  manual  <RGL  11.4). 

These  findings  indicated  that  some  differences  existed 
between  the  manuals.  However,  when  taken  as  a  whole,  manual 
sets  were  found  to  be  not  significantly  different.  Both 
individual  manuals  and  manual  sets  exhibited  positive  and 
negative  quantitative  and  qualitative  characteristics,  and 
their  differences  averaged  out.  The  only  differences  that 
seemed  appreciable  were  the  point  size  and  page  layout 
comparisons.  The  DMVRs  appeared  to  be  more  consistent  and 
consistently  better,  in  point  size  comparisons.  However,  as 
MIL-M-38784B  stipulates  that  ”t DMVRs]  shall  be  such  as  to 
provide  for  a  minimum  final  letter  size,  when  printed,  of  8 
paints”,  <32a: 3. 6. 1 . 1 )  and  this  research  study  showed  that 
60%  of  the  manuals  were  below  that  standard,  DMVRs  could  be 
improved  by  having  sizes  of  at  least  8  points.  Commercial 
manuals  appeared  to  be  more  consistent  and  consistently 
batter  in  page  layout  comparisons.  This  finding  seemed 
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strange,  since  MIL-M-38784B  stipulates  that  ’’unless 
otherwise  specified,  manuals  shall  be  prepared  double¬ 
column”.  <32a:3.2.2)  Only  3  of  15  DMVRs  had  a  double  column 
layout.  <See  page  67  for  the  other  results. ) 

The  RGL  values  ascended  from  trucks  to  aircraft  to 
radios,  and  this  rise  in  RGL  value  held  logically  over 
commodities,  parallelling  technological  difficulty  between 
commodities.  It  was  surprising  to  find  that  the  commercial 
overhaul  repair  sector’s  manuals  generally  had  lower  RGLs 
than  did  the  Depot's,  in  light  of  Duffy’s  research 
showing  that  commercial  manuals  tend  to  have  only  a  cursory 
interest  in  usability.  <46:386)  It  would  seem  logical  that 
if  the  commercial  sector  did  not  particularly  care  about 
having  a  low- RGL  value  manual,  it  would  not  have  had  one. 

The  overall  finding,  that  there  was  no  difference 
between  the  sets,  lends  doubt  to  Gringas’  assertion  that  the 
"simplified  Code  English"  of  many  manuals  (i.e.,  Caterpillar 
manuals)  is  significantly  different  from  the  English  of 
DMVRs.  <47:25)  In  the  user  test  portion  of  this  research 
study,  Caterpillar  "Code  English”  was  compared  to  the 
language  in  other  commercial  and  Depot  manuals,  and  although 
Caterpillar  manuals  seemed  very  easy  to  use,  there  appeared 
to  be  no  differences  in  the  language  used. 

Duffy  claimed  that  the  commercial  world  generally  lacked 
appreciation  for  manual  usability,  but  Depots  have  some 
appreciation  for  it.  <46:386)  However,  three  commercial 
manuals  surveyed  included  usability  questionnaires  in  their 
texts,  whereas  no  DMVRs  had  them- -but  every  DMVR  did  have 
some  sort  of  short  statement  encouraging  user  comments,  (see 
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Appendix  J)  No  other  commercial  manuals  made  this  attempt. 

The  overall  findings  seemed  to  support  the  TRV  study, 
which  also  discovered  few  <lf  any)  significant  differences 
between  Depot  and  commercial  aircraft  overhaul  manuals. 
(16:2)  Additionally,  both  sets  of  manuals  appeared  to  have 
similar  faults.  The  user  test  portion  of  this  research 
study  showed  that  commercial  and  Depot  aircraft  manuals  had 
approximately  the  same  style,  layout,  format,  and  even  vied 
Jointly  for  having  the  most  imprecise  words  and 
ungrammatical  sentences.  (See  Appendix  J) 

By  strict  count  of  this  user  test  data  only,  commercial 
manuals  exhibited  more  characteristics  positive  to  reader 
motivation  than  DMVRs  did,  but  both  were  equal  for  negative 
motivators.  Since  negative  characteristics  received  an 
unweighted  value  of  -1,  and  positive  characteristics 
received  an  unweighted  value  of  +1,  Depots  received  a  user 
test  score  for  RGLs  (in  this  viewing)  of  -1,  and  commercial 
manuals  received  a  score  of  0.  These  results  are  not 
appreciably  different. 

1 1 lustrat ions. 

Findings.  The  second  hypothesis  of  the  study  stated 
that  there  was  no  significant  difference  between  the 
commercial  and  Depot  manuals’  readability,  and  was  measured 
by  comparing  the  frequency  of  manual  illustrations. 
Determining  the  validity  of  this  hypothesis  required 
comparison  of  Illustration  Frequency  data  collected  from  the 
sample  population.  The  results  are  summarized  in  Table  3, 
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which  compares  the  average  overall  illustration  frequency 
for  the  commercial  and  Depot  commodities  of  truck  engines, 
aircraft  engines,  and  radio  receivers/transmitters.  The 
individual  illustration  scores  were  recorded  as  a  number  of 
illustrations  per  page.  The  Table  shows  that  the 
average  illustration  score  for  commercial  truck  engines  was 
1.1  higher  than  the  Depot  average;  for  commercial  aircraft 
engines  it  was  0.2  lower  than  the  Depot  average;  and  for 
commercial  radio  receivers/transmitters  it  was  0.  1  lower 


than  the  Depot  average. 


Table  3 

ILLUSTRATION  FREQUENCY  DATA  SUMMARY 


Anova  test.  To  test  hypothesis  two,  the  ANOVA  test  was 
performed  to  measure  significant  differences.  (3a: 503-508) 
Table  4  summarizes  the  ANOVA  test  data.  The  Table  shows  the 
data  collected  from  the  Grand  Means  and  the  F  statistic 
calculations  (see  appendix  F).  The  critical  value  in  the: 
Table  was  derived  from  the  table  in  Appendix  H. 
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Table  4 

Two  Vay  ANOVA  Data  for  Illustrations 


Block 

MILITARY 

MANUAL 

COMMERCIAL 

MANUAL 

ROW  TOTALS 

1 

ROW  MEANS  X 

Truck 

Engines 

(TE) 

0.5 

1.6 

2.  1 

1. 05 

Aircraft 

Engines 

(AE) 

0.6 

0.4 

1.0 

0.5 

Radio 

Receivers 

(RR) 

0.5 

0. 4 

0. 9 

0. 45 

Column 

Totals 

2. 4 

1.6 

Co  3 umn_ 
Mea..s  X 

0.6 

0. 53 

Grand  Mean  X  =  (X,  +  X,)  /  2  Grand  Mean  =  (XTe+  X^+  ^  /  3 
(columns)  =  0.665  (rows)  =0.67 


For  formulas,  see  Appendix  F. 

The  Critical  Value  from  Appendix  H,  with  1  and  4  Degrees  of 
Freedom,  equals  7.71. 


Hypothesis,  With  1  and  4  degrees  of  freedom  at  the  0.05 
level  of  significance,  the  critical  value  of  7.71  was 
obtained  from  Appendix  H.  If  the  value  calculated  using  the 
Two-Way  AHOVA  test  was  less  than  or  equal  to  the  critical 
value,  the  hypothesis  should  be  accepted  <fail  to  reject) 
<3a:507>.  The  calculated  values  of  0.413  for  treatments  and 
0.826  for  blocks  were  both  much  lower  than  the  critical 
value.  Therefore,  the  stated  hypothesis  was  accepted.  There 
were  no  significant  differences  in  the  frequency  of 
illustrations  between  the  commercial  overhaul  manuals  and 
Depot  overhaul  manuals. 

Related  Characteristics.  The  following  characteristics 
of  commercial  manuals  represent  the  opinion  of  a  manual  user 
<the  researcher)  recording  aspects  of  the  manuals  especially 
negative  (or  positive)  to  reader  motivation  (for  additional 
characteristics,  see  Appendix  J).  All  of  the 
characteristics  in  this  set  represent  the  initial  of  two 
viewings,  and  record  only  observations  relating  to 
illustration  frequency.  (The  following  two 
'•characteristics’’  of  commercial  manuals  were  especially 
negative  to  reading  motivation). 

1.  Caterpillar  manuals  did  not  have  numbered 
illustrations  or  figures. 

2.  Allison  manuals  had  no  illustration  index,  and 
tables,  figures  and  illustrations  were  scattered  throughout 
the  text. 

The  following  characteristics  of  Depot  manuals  were 
especially  negative  to  reading  motivation: 
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1. 


Illustrations,  no  matter  how  small,  were  placed  on 


a  page  their  own.  This  caused  separations  of  text  and 
illustrations. 

2.  DXVRs  used  only  black  and  white,  line  drawings. 

3.  DXVR  pages  were  not  headed  by  a  title  which  defined 
the  work  being  done.  Since  there  were  no  illustrations  on 
the  same  page  as  the  text,  it  was  very  hard  to  keep  the  many 
different  operations  within  one  manual  separated,  especially 
since  illustrations  separated  from  text  caused  the  reader  to 
have  to  turn  a  lot  of  pages. 

4.  Some  DXVRs  had  no  table  of  illustrations,  though 
this  table  is  required  by  XIL-X-38784B. 

The  fallowing  characteristics  of  corafrercial  manuals  were 
especially  positive  to  reader  motivation: 

1.  Cummins  manuals  had  the  same  illustration  format 
for  each  page:  one  repair  step,  one  illustration.  Cummins 
also  placed  a  multi-language  pictorial  instruction  symbol  by 
each  picture,  making  pictures  and  instructions  not  only  very 
simple,  but  also  more  understandable  in  multi-lingual 
settings. 

2.  Detroit  manuals  had  only  high  resolution  black  and 
white  photographs  for  illustrations. 

3.  Caterpillar  and  GE  manuals  had  some  color  in  their 
i 1 lustrat ions . 

The  following  characteristics  of  Depot  manuals  were 
especially  positive  to  reader  motivation: 

AVSCOX  DXVB  55-2835-209  was  written  in  double 
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1. 


column  paragraphs  with  illustrations  interspersed  with  text. 

2.  Ho  illustration  was  reduced  in  size  simply  to  make 
it  fit  with  the  text. 

3.  CECOM  DMVR  11-5805-722-34  (and  the  TM>  had  double 
column  paragraphs  and  black  and  white  photograph 
illustrations  interspersed  with  text.  Its  TM  had  higher 
quality  type  and  illustrations  than  the  DMVR  did. 

Analysis  for  Illustration  Frequency 
Use  of  the  Two-Way  ANOVA  statistical  test  on  the 
illustration  data  collected  from  DMVRs  and  comparable 
commercial  overhaul  manuals  revealed  that  there  was  no 
statistical  difference  between  the  two  manual  sets.  The 
largest  difference  between  any  set  of  manuals  occurred 
between  commercial  and  Depot  truck  engines  <1.1  more 
illustrations  per  page  in  commercial  manuals).  The  smallest 
difference  was  between  commercial  and  Depot  radio  receivers/ 
transmitters  <0.1  more  illustrations  per  page  in  Depot 
manuals).  The  greatest  variance  within  this  test  was 
between  a  commercial  truck  engine  manual  <1.9)  and  a  Depot 
truck  engine  manual  <0.3).  The  highest  and  lowest 
illustration  frequency  were,  respectively,  a  commercial 
truck  engine  manual  <1.9  illustrations  per  page),  and  a 
commercial  radio  manual  <0.1  illustrations  per  page). 

These  differences  (although  statistically  insignificant) 
indicated  that  commercial  and  Depot  truck  engine  manuals  were 
the  most  variable  and  different.  The  qualitative  test 
supported  this  finding,  because  it  seemed  to  indicate  that 
these  manuals  were  the  most  positively  motivating,  being 
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impressively  and  professionally  laid  out  and  instructive. 

Apparently,  commercial  manual  writers,  not  constrained 
by  rigid  manual  specifications,  included  illustrations  to 
clarify  text  (the  purpose  of  illustrations),  and  not  simply 
to  fulfill  a  specification  mandate.  It  appeared  to  be  true 
for  illustrations  in  DMVRs  that  they  often  appeared  to  be 
cursory,  almost  an  afterthought.  In  this  respect,  the  MRSA 
study  contention  that  DMVRs  could  be  improved,  (8: iv)  seemed 
to  be  supported.  DMVRs  could  have  better  illustrations  such 
as  good  photographs  and  illustrations  closer  to  the  text  and 
interspersed  with  the  text.  Commercial  manuals  often  had 
these  characteristics. 

It  was  surprising  to  find  that  commercial  aircraft 
manuals,  usually  published  through  high  technology,  modern, 
financially  stable  companies  and  made  for  expensive,  tightly 
controlled  equipment,  did  not  make  better  or  greater  use  of 
illustrations.  Illustrations  can  rivet  and  focus  the 
attention  of  any  mechanic,  and  therefore  are  a  great  means 
of  multiplying  the  effectiveness  of  any  manual.  It  was  not 
surprising  to  find  that  the  illustration  frequency  rose 
respectively  from  low  to  high  in  radios  and  trucks.  This 
progression  seemed  to  correlate  well  with  the  conceptual 
difficulty  of  the  respective  repair  technologies. 

By  strict  count  (for  this  data  section  only),  commercial 
manuals  exhibited  more  characteristics  positive  to  reader 
motivation  than  DMVRs  did,  and  fewer  negative 
characteristics.  Since  negative  characteristics  received  an 
unweighted  value  of  -1,  and  positive  characteristics 
received  an  unweighted  value  of  +1,  Depots  received  a  user 
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teat  score  for  illustration  frequency  (in  this  viewing)  of 
-1,  and  commercial  manuals  received  a  score  of  +1.  The  test 
scores  did  not  appear  to  be  appreciably  different. 

Cues. 

Findings.  The  third  hypothesis  of  the  study  stated 
that  there  was  no  significant  difference  between  the 
commercial  and  Depot  manuals'  readability,  measured  by  the 
manuals’  use  of  aids  to  understanding  (cues) .  Testing  this 
hypothesis  required  comparing  cue  data  collected  from  the 
sample  population.  These  data  are  summarized  in  Table  5. 

This  Table  compares  the  average  overall  cue  frequency  for 
the  commercial  and  Depot  commodities  of  truck  engines, 
aircraft  engines,  and  radio  receivers/transmitters.  The 
individual  cue  scores  represent  the  number  of  cues  per  page. 
Larger  values  are  better  than  smaller  values.  The  Table 
shows  that  the  average  cue  score  for  commercial  truck 
engines  was  0.8  higher  than  the  Depot  average;  for  commercial 
aircraft  engines  it  was  0.3  lower  than  the  Depot  average; 
and  for  commercial  radio  receivers/transmitters  it  was  0.8 
higher  than  the  Depot  average. 
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Table  5 

CUE  FREQUBHCY  DATA  SUMMARY 


Anova  test.  To  test  hypothesis  two,  the  ANOVA  test  was 
performed  to  measure  significant  differences.  (3a: 503-508) 
Table  6  summarizes  the  AITOVA  test  data.  The  Table  shows  the 
data  collected  from  the  Grand  Means  and  the  F  statistic 
calculations  (see  appendix  F  for  the  formulas).  The 
critical  value  in  the  Table  was  derived  from  the  table  in 
Appendix  H. 
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Table  6 

Two  Vay  AITOVA  Data  for  Cues 


Block 

MILITARY 

MANUAL 

COMMERCIAL 

MANUAL 

ROV  TOTALS 

ROW  MEANS  X 

Truck 

Engines 

(TE) 

4.0 

4.8 

8.8 

4.4 

Aircraft 

Engines 

(AE> 

3.  8 

3.5 

7.3 

3.65 

Radio 

Receivers 

(RR) 

4.8 

5.6 

10.  4 

5.2 

Column 

Totals 

12.6 

13.9 

Column_ 
Means  X 

4.2 

4.63 

Grand  Mean  X  =  (X,  +  X*)  /  2  Grand  Mean  =  (X  +  X  +  X)  /  3 
(columns)  =  4.415  (rows)  =  4.417  ^ 


Source  of 
Variation 

Sum  of  Squares 

Degrees  of 
Freedom 

Mean 

Square 

Test 

Statistic 

Treatments 
Military  & 
Commercial 

0.276 

1 

0.276 

Ft  =1. 34 

Blacks 

TE.  AE.  RR 

2. 40 

2 

1.20 

CO 

oo 

in 

n 

Error 

0.  412 

2 

■ 

Total 

3.087 

5 

For  formulas,  see  Appendix  F. 

The  Critical  Value  from  Appendix  H,  with  1  and  4  Degrees  of 
Freedom,  equals  7.71. 
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Hypothesis.  With  1  and  4  degrees  of  freedom  at  the  0.05 
level  of  significance,  the  critical  value  of  7.71  was 
obtained  from  Appendix  H.  If  the  value  calculated  using  the 
Two-Way  AITOVA  test  was  less  than  or  equal  to  the  critical 
value,  the  hypothesis  should  be  accepted  (fail  to  reject) 

(3a: 507).  The  calculated  values  of  1.34  for  treatments  and 
5.83  for  blocks  were  both  lower  than  the  critical  value. 
Therefore,  the  stated  hypothesis  was  accepted.  There  were 
no  significant  differences  in  the  frequency  of  cues  between 
the  commercial  overhaul  manuals  and  Depot  overhaul  manuals. 

Related  Characteristics.  The  following  characteristics 
of  commercial  manuals  represent  the  opinion  of  a  manual  user 
(the  researcher)  recording  aspects  of  the  manuals  especially 
negative  (or  positive)  to  reader  motivation  (for  additional 
characteristics,  see  Appendix  J).  All  of  the 
characteristics  in  tnis  set  represent  the  initial  of  two 
viewings,  and  record  only  observations  relating  to  cue 
frequency.  (The  first  four  characteristics  were  especially 
negative  for  commercial  manuals. ) 

1.  Allison  manuals  used  cues  mostly  on  page  headers 
and  footers,  leaving  out  cues  elsewhere  in  the  text. 

2.  Perkins  figure  numbers  were  hard  to  visually  pick 
out  from  the  text. 

3.  Perkins  manuals  used  vertical  lines  in  the  column 
to  highlight  text,  but  these  were  so  unobtrusive,  they  did 
not  stand  out  readily. 

4.  Detroit  manuals  used  few  cues,  except  to  underline 
the  word  note ■ 
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Depot  manuals  had  no  especially  negative  characteristics 


recorded  from  this  viewing. 

The  following  characteristics  for  commercial  manuals 
were  especially  positive  to  reader  motivation: 

1.  Note  and  caution  references  in  Cummins  manuals  were 
especially  easy  to  see,  because  the  double  column  paragraphs 
were  as  wide  as  the  note,  and  a  reader  could  not  gloss  over 
it  to  get  on  to  repair  information. 

2.  Detroit  manuals  made  use  of  lots  of  italics,  which 
stood  out  well. 

3.  Cummins  had  manuals  written  in  mainly  picture/ 
symbolic  language,  with  symbols  explained  in  the  index  in 
four  major  languages. 

4.  All  aviation  manuals  used  many  varied-shaped  cues. 

The  following  characteristics  of  Depot  manuals  were 
especially  positive  to  reader  motivation: 

1.  AVSCOM  used  more  varied  shapes  of  cues  than  any 
other  DMWRs. 

2.  CECOM  DMVR  11-5805-722-34  used  cues  more  than  any 
other  manual,  and  had  varied-shaped  cues. 

Analysis  for  Cue  Frequency 

Use  of  the  Two-Way  AHOVA  statistical  test  on  the  cue 
data  collected  from  DMWRs  and  comparable  commercial  overhaul 
manuals  revealed  that  there  was  no  statistical  difference 
between  the  two  manual  sets.  The  largest  difference  between 
any  cat  of  manuals  occurred  in  a  tie  between  commercial  and 
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Depot  truck  engines  and  commercial  and  Depot  radios  <0.8 
more  cues  per  page  In  commercial  manuals).  The  smallest 
difference  was  between  commercial  and  Depot  aircraft  manuals 
<0.3  more  cues  per  page  in  Depot  manuals).  The  greatest 
variance  between  nanuals  was  between  a  Depot  radio  manual 
<9.2  cues  per  page)  and  another  Depot  radio  manual  <2.7  cues 
per  page).  The  largest  variance  between  the  two  repair 
sectors  was  between  commercial  and  Depot  radio  manuals  <7.4 
and  2.7  cues  per  page,  respectively). 

DMVR  radio  manuals  apparently  had  the  greatest 
variability  in  cue  frequency,  between  their  own  manuals  and 
with  commercial  manuals.  The  qualitative  test  also 
indicated  that  all  radio  manuals,  on  the  average,  contained 
the  least  specific  repair  procedures. 

It  was  surprising  to  find  that  commercial  aircraft 
manuals,  usually  published  through  high  technology,  modern, 
financially  stable  companies  and  made  for  expensive,  tightly 
controlled  equipment,  did  not  make  better  or  greater  use  of 
cues.  Cues  are  inexpensive  and  an  easy  effective  means  to 
multiply  the  effectiveness  of  any  manual. 

The  GAO  study  claim  that  DMVRs  had  ’’Compl icated 
descript ions. .. poorly  explained  procedures.  .. I  and)  hard  to 
locate  information”  <12:5)  was  borne  out  by  the  user  test 
findings.  However,  since  there  were  no  apparent  differences 
in  the  manuals,  commercial  and  Depot  manuals  were  Just  as 
likely,  therefore,  to  exhibit  the  same  faults.  The  GaO 
assertion  that  there  was  a  great  ’’uniformity  between  DMVR 
manuals”  <12:27)  appeared  to  hold  true. 

By  strict  count  <of  this  first  user-test  ’’viewing”), 
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commercial  manuals  exhibited  more  characteristics  negative 
to  reader  motivation  than  did  DKVRs,  and  more  positive 
characteristics,  also.  Since  negative  characterist ics 
received  an  unweighted  value  of  -1,  and  positive 
characteristics  received  an  unweighted  value  of  +1,  Depots 
received  a  user  test  score  for  cue  frequency  (in  this 
viewing)  of  +2,  and  commercial  manuals  received  a  score  of 
+0.  The  test  scores  did  not  appear  to  be  appreciably 
different . 

General  Characteristics. 


The  following  set  of  data  are  general  characteristics  of 
the  manuals  which  stood  out  as  either  especially  positive  or 
negative  to  reader  motivation.  They  are  separate  from  the 
lists  of  positive  and  negative  characteristics  in  the  other 
sections,  because  they  did  not  closely  relate  to  RGLs, 
illustrations  or  cues.  These  were  included  because  they  were 
collected  in  the  initial  viewing  of  the  manuals.  They  were 
also  included  as  a  help  to  other  researchers  should 
subsequent  related  research  be  done.  (See  Appendix  J  for 
other  characteristics).  The  first  five  characteristics 
are  especially  negative  from  commercial  manuals: 

1.  Xack  manuals  used  many  ’’non-specific”  words,  such 
as  "suitable”  and  "particular”. 

2.  The  Caterpillar  manual  had  so  few  words,  it  often 
took  more  than  one  page  to  collect  150-word  sections.  The 
repair  descriptions  were  so  short,  they  did  not  explain 
procedures  as  completely  as  other  manuals  did. 
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3.  The  Mack  manual  was  very  segmented,  and  this  made 
It  hard  to  read,  because  a  reader  had  to  flip  through 

to  follow  a  repair  procedure. 

4.  The  newest  aviation  manuals  were  written  in  single 
column  paragraphs,  which  contributed  to  eye  fatigue. 

5.  General  Electric  and  Allison  manuals  had  a  very 
confusing  pagination  system,  not  obviously  sequential,  with 
no  easy  tabs. 

Depot  manuals  had  no  especially  negative  characteristics 
to  be  reported  from  this  viewing. 

The  following  positive  characteristics  of  rnwinarrial 
manuals  were  recorded: 

1.  Detroit  manual  paper  was  very  thin,  so  the  manuals 
seemed  very  light  to  hold  and  manipulate. 

2.  Cummins  manuals  discussed  the  use,  scope  and 
orientation  of  an  illustration  with  respect  to  tha  equipment 
being  repaired.  It  also  had  a  very  good  section  on  "how  to 
use  this  manual”,  giving  four  general  steps  to  follow  for 
normal  situations. 

3.  Detroit  manuals  had  pages  throughout  for  ’’shop 
notes. ” 

4.  Detroit  and  Caterpillar  manuals  provided  the  order 
numbers  for  special  tools,  to  help  in  ordering. 

5.  Aviation  and  radio  manuals  had  sections  on  mounting 
and  installation  instructions  not  found  in  other  manuals. 

6.  Motorola  and  E.  F.  Johnson  manuals  each  had  ’’Service 
Manual  Questionnaires”  which  asked  potential  manual  users 
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specific  questions,  an  a  mail-in  card,  on  ways  to 
improve  manual  usability. 


Depot  manuals  had  no  especially  positive  characteristics 
recorded  from  this  viewing. 

By  allotting  points  to  the  foregoing  positive  and 
negative  characteristics  as  previously,  commercial  texts 
accumulated  (for  the  first  ’’viewing)  +1  points  overall,  and 
Depot  manuals  accumulated  no  points.  This  difference  did 
not  seem  appreciable. 

User  Test  Results. 

Vhen  all  of  the  characteristics  from  all  sections  with 
their  simple  value  designations  were  taken  together,  the 
overall  values  were  essentially  equal,  as  shown  in  Table  7. 

Table  7 

User  test  results 


Commercial 

Depot 

Commercial 

Depot 

Positive 

Char. 

% 

Positive 

Char . 

% 

Negative 

Char. 

% 

Negative 

Char . 

% 

37 

37% 

38 

40% 

63 

63% 

57 

60% 
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These  data  tended  to  indicate  that  the  manual  sets  were  both 


very  similar  in  the  power  to  motivate  reading,  and  that  both 
had  essentially  the  same  proportion  of  positive  and  negative 
characteristics. 

It  was  surprising  to  find  that  the  Depot  manuals  had 
less  bad  and  more  good  characteristics  than  commercial 
manuals  did.  The  DESCOM  user  test  results  <36-42),  which 
indicated  that  DMVRs  have  been  generally  less  usable  than 
commercial  manuals,  appear  to  slightly  contradict  these 
findings.  The  fact  that  both  sets  had  more  negative 
characteristics  than  positive  ones  is  understandable  in 
light  of  the  usual  low  emphasis  put  on  usability.  <46:386) 
But  that  the  fact  that  the  negative  characteristics  occurred 
twice  as  frequently  as  positive  characteristics  was 
surprising.  With  such  a  large  sample  population  as  was  used 
in  this  study,  equal  numbers  of  positive  and  negative 
characteristics  would  seem  more  likely  than  double  the 
occurrence  of  negative  characteristics. 

These  various  tests,  taken  together  or  separately, 
contradict  the  DESCOM  assertion  that  DMVRs  are  more 
’'complex*'  than  commercial  manuals  and  have  "[more)  extensive 
[difficult]  work  performance  instructions”  <36:3-1),  and  are 
therefore  not  "fully  competitive  with  contract  [commercial! 
Sources  of  Repair.”  <40:1)  The  results  of  these  varied  test 
showed  no  significant  differences  between  the  manuals. 
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CHAPTER  V 


SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 


Problem 


The  purpose  of  this  study  was  to  compare  classes  of 
equipment  overhaul  manuals  of  the  US  Army  Depot  maintenance 
system  with  corresponding  classes  of  manuals  of  the  US 
commercial  equipment  maintenance  system  to  determine  if 
Depot  overhaul  manuals  were  relatively  more  difficult  to 
use.  The  first  subproblem  was  to  determine  the  relative 
simplicity  of  the  manuals,  as  in  their  ease  of  readability. 
The  second  subproblem  was  to  determine  the  relative  number 
of  illustrations  per  manual  between  the  manual  sets.  The 
third  and  last  subproblem  was  to  determine  the  frequency  of 
the  use  of  any  aids  to  understanding  in  the  texts. 

Procedure 


The  procedure  used  in  this  study  was  to  select  fifteen 
Depot  and  fifteen  commercial  overhaul  manuals  from  three 
representative  equipment  categories,  and  test  the  degree  of 
similarity  in  the  manuals  for  the  following:  Reading  Grade 
Levels,  use  of  illustrations  and  instructional  cues,  and  the 
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"reading  motivation"  affect  upon  manual  users.  The  data 
were  collected  and  tabulated,  and  a  Two-Vay  ANOVA 
statistical  test  was  performed  on  the  results  of  three  of 
the  tests  to  determine  if  they  were  statistically  different. 
The  remaining  comparison  test,  a  two  part,  modified 
usei — test,  collected  general  observations  and 
characteristics  of  the  manuals.  The  observer  was  one 
unbiased  reader  who  critically  observed  all  of  the  texts  for 
any  traits  that  motivated  or  demotivated  reading. 

The  first  three  tests  were  designed  to  answer  the  three 
hypothesis  of  the  study.  The  modified  user  tests  were 
included  to  increase  the  locus  of  tests  to  include  some 
which  were  qualitative  in  nature.  Their  inclusion  was  seen 
a&  a  way  to  help  substantiate  the  findings  of  the  other 
tests.  They  would  also  be  a  significant  help  to  the 
interpreting  of  research  finding  differences. 

Major  Findings 

The  major  findings  of  this  research  study  were  as 
follows: 

1.  There  were  no  statistically  significant  differences 
between  the  Heading  Grade  Level  (RGL)  of  commercial  overhaul 
manuals  and  Depot  overhaul  manuals.  Individual  differences 
existed,  but  these  differences  were  not  significant  when  all 
manuals  were  compared. 

2.  There  were  no  statistically  significant  differences 
in  the  frequency  of  illustrat inns,  between  commercial 
overhaul  manuals  and  Depot  overhaul  manuals.  However,  two 
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commercial  manual  Illustration  practices  stood  out  as 
superior: 

a.  commercial  manuals  had  multiple,  high  resolution, 
black  and  white  photograph  illustrations  interspersed  with 
supporting  text  descriptions.  DMVHs  almost  never  used 
photographs.  Consequently,  DMVR  illustrations  were  almost 
always  separated  to  some  degree  from  their  supporting  text. 

b.  Commercial  manual  illustrations  seemed  to  lie  much 
closer  to  their  supporting  texts.  Though  this  finding  was 
not  permitted  to  be  quantified  and  subjected  to  statistical 
verification  <by  the  research  study’s  restricted  scope),  it 
was  also  the  opinion  of  other  researchers  of  the  related 
literature.  <12:5) 

Much  research  has  been  completed  upholding  the 
superiority  of  photographs  over  line  drawings  (52:739  and 
0:5)  and  showing  the  benefits  of  illustrations  interspersed 
with  supporting  text.  <62:422  and  12:5)  DMVH  writers  should 
follow  these  recommendations  to  improve  reading  and 
understanding  of  their  manuals. 

3.  There  were  no  statistically  significant  differences 
in  the  frequency  of  the  use  of  cues  between  commercial 
overhaul  manuals  and  Depot  overhaul  manuals.  However,  of 
all  the  tests,  the  calculated  ANOVA  statistical  test  values 
for  cue  frequency  were  the  closest  to  the  critical  F  table 
values.  There  appeared  to  be  a  greater  difference  between 
the  manual  sets  for  cue  frequency  than  for  the  other  tests. 
This  difference  borders  si gni f icancc ,  but  still  does  not 
penetrate  a  true  significance  range. 

In  general,  cues  help  readers  read  and  understand 
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better.  DMVR  writers  should  make  better  use  of  cues  to 
improve  reading  and  understanding. 

4.  The  user  tests  seemed  to  indicate  that  the  manual 
sets  were  both  similar  in  their  powers  to  motivate  reading, 
and  had  essentially  the  same  proportion  of  positive  and 
negative  characteristics.  DMVRs  had  slightly  less  negative 
and  more  positive  characteristics  than  commercial  manuals 
had.  In  both  manuals,  negative  characteristics  occurred 
approximately  twice  as  frequently  as  positive 
characteristics  did. 

5.  The  only  differences  that  appeared  to  be  significant 
(though  perhaps  only  slightly),  were  in  the  areas  of  point 
size  and  page  layout  comparisons.  DMVRs  appeared  to  have 
consistently  larger  letter  point  sizes  than  commercial 
manuals  had,  and  with  a  lower  variance  of  sizes.  However, 
the  DMVRs  studied  also  had  60%  of  their  letter  point  sizes 
below  the  regulati  n  standard  of  size  8.  Larger  point  sizes 
are  easier  to  read  than  smaller  point  sizes.  The  commercial 
manuals  used  (the  preferable)  double  column  page  layout  67% 
of  the  time,  whereas  DMVRs  used  double  column  layouts  only 
19%of  the  time.  This  was  in  spite  of  the  fact  that  DMVRs 
were  required  by  regulation  to  use  a  double  column  format. 

Conclusions 


It  appears  that  the  Depot  and  commercial  manuals  were 
not  Inherently  harder  or  easier  to  read  (judged  by  RGL; ,  to 
conceptualize,  (Judged  by  illustration  frequency),  or  to 
learn  from  (judged  by  cue  frequency),  for  mechanics  assumed 
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to  have  equal  levels  of  grade  level  Instruction  and  mental 
maturity.  The  sets  appeared  to  have  essentially  equal  user 
test  reading  motivation  characteristics.  The  only  noticeable 
differences  seemed  to  have  appeared  in  the  user  test 
observations  of  point  size  and  text  layout.  DKVRs  had 
generally  larger  letter  point  sizes,  but  generally  worse 
double  column  text  layout  style. 

The  values  generated  by  the  RGL  test  could  be  larger  by 
a  factor  of  7  before  any  significance  could  be  seen  in  RGL 
differences.  The  values  generated  by  the  illustration  test 
could  also  be  larger  by  a  factor  of  7  before  any  significant 
difference  could  be  seen  in  illustration  frequency. 

However,  cue  frequency  test  values  larger  by  a  factor  of 
only  1.2  points  could  generate  a  significance  difference  in 
cue  frequencies.  This  means  that  the  individual  manuals  in 
the  RGL  and  Illustration  tests  each  had  very  small  variances, 
but  the  variances  between  individual  manuals  for  the  cue 
test  were  larger  (but  not  significantly  different). 

These  various  tests,  taken  together  or  separately, 
appear  to  contradict  the  DESCOM  assertion  that  DKVRs  are 
more  "complex"  (36:3-1)  than  commercial  manuals,  have 
"[more]  extensive  [difficult]  work  performance  instructions" 
(36:3-1),  and  therefore  are  not  "fully  competitive  with 
contract  [commercial]  Sources  Of  Repair".  (40:1)  By  these 
tests  there  were  no  significant  differences,  and  the  small 
positive  and  negative  differences  essentially  cancelled  each 
other  out. 
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Recommendw-M 


The  following  recommendations  were  made  by  the 
researcher,  based  on  the  overall  findings: 

1.  DESCOH  and  AMC  should  consider  the  findings 
outlined  in  this  study  before  making  any  final  decisions  to 
retire  or  modify  the  DMVR  library,  if  the  decision  would  be 
based  heavily  on  the  assumption  that  Depot  and  commercial 
manuals  are  significantly  different  with  respect  to  ease  of 
use. 

2.  More  extensive  tests  should  be  designed  or 
collected  which  look  for  potential  differences  between  Depot 
and  commercial  overhaul  manuals.  Significant  differences 
could  be  used  in  recommending  changes  in  the  manuals.  It  is 
recommended  that  other  Logistics  Management  Masters  Degree 
students  make  the  study,  because  they  both  have  six  months 
of  uninterrupted  time  to  make  the  complex  study,  they  have 
extensive  library  access  at  Ft  Lee,  VA,  and  this,  their 
research  project,  cannot  be  postponed — it  must  be  completed 
within  a  six-month  period. 

3.  Other  user  tests  should  be  performed  employing  both 
experienced  graders  and  many  manual- inexperienced  test 
takers.  The  findings  should  be  compared  to  the  findings  in 
this  study. 

4.  Exact  measurements  comparing  the  actual  degree  of 
illustrations/text  separations  should  be  made. 

5.  The  scope  of  the  subsequent  studies  should  be 
enlarged  to  allow  inclusion  of  other  English  language 
maintenance  manuals  from  other  countries,  such  as  from 
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Japan.  The  best  ideas  from  all  nations  should  be  used  if 
possible. 

6.  As  a  minimum,  the  following  characteristics 
addressed  in  the  user  tests  should  be  fallowed  up  with  more 
research: 

a.  Compare  the  two  sets  for  the  relative  frequency 
of  step-by-step  repair  procedures,  where  each  step  is 
clearly  set  apart  from  adjacent  steps  by  cueing  devices. 

b.  Discover  which  DMVRs  do  not  have  illustration 
indexes. 

c.  Discover  the  degree  of  separation  of 
illustrations  from  their  supporting  texts,  in  DMVRs. 

d.  Explore  the  benefits  from  using  brief  statements 
with  each  illustration  which  describes  its  scope  and  general 
orientation  in  the  repair  and  on  the  equipment. 

e.  Explore  the  benefits  of  including  "service 
questionnaires"  and  blank  "shop  notes”  pages  in  DMVRs. 

f.  Explore  the  benefits/liabilities  of  the  use  of 
active/passive  voice  in  repair  procedures  in  DMVRs. 

g.  Explore  the  reception  among  Depot  mechanics  of 
the  inclusion  of  part  and/or  tool  order  numbers  in  DMVRs. 

7.  A  study  of  the  cost  benefits/liabilities  of 
swapping  DMVRS  for  some  other  form  of  manuals  should  be 
conducted.  A  study  should  also  be  made  of  the  benefits/ 
liabilities  of  editing  existing  technical  data  packages  to 
make  them  conform  to  DMVR  format.  This  research  should  help 
answer  the  question  whether  trading  DMVRs  for  commercial 
manuals  may  be  more  costly  than  benficial  and  whether 
editing  commercial  manuals  to  make  them  into  DMVRs  could  be 
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an  unneeded  duplication. 

8.  DXVRs  should  make  more  use  of  the  following  reading 
"motivators",  in  order  to  have  better  (and  possibly  better 
utilized)  manuals: 

a.  Use  double  column  page  style. 

b.  Use  photograph  illustrations,  with  at  least  one 
color  other  than  black. 

c.  Use  more  and  better  reading  cues  which  help 
understanding . 

d.  Intersperse  illustrations  better  with  supporting 

text . 

e.  Design  a  Depot  manual  to  have  a  low  RGL  if  its 
overall  readability  can  be  verified  as  being  high,  by  user 
tests. 

f.  Ensure  that  letter  point  size  is  at  least  7,  but 
if  possible,  always  8. 
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20  Mar  1990 


SUBJECT:  Request  to  study  some  commercial  manuals 

TO:  Superior  Diesel  Service 
2106  E  Main  Street 
Richmond  Virginia 

1.  The  Army  Logistics  Management  College  in  conjunction 
with  the  Florida  Institute  of  Technology  is  sponsoring  a 
study  to  suggest  ways  to  improve  the  Army  equipment  overhaul 
manuals,  called  Depot  Maintenance  Work  Requirements  (DMVR) . 
This  letter  requests  permission  to  look  at  the  form  and 
content  of  copies  of  selected  commercial  manuals  at  your 
facility  in  the  month  of  April  or  May,  1990,  to  make  the 
comparisons. 

2.  I  am  the  principal  researcher.  My  name  is  Captain  Mark 
Newell,  SSN  550-25-9081.  I  am  stationed  at  Ft  Ben  Harrison, 
Ind,  with  duty  assignment  at  Ft  Lee,  Va,  until  20  June  90. 

My  mailing  address  is  363-A  Coral  Sea  Dr,  Ft  Lee,  Va,  23801. 
My  phone  at  which  I  can  be  reached  is  (804)  733-7069. 

3.  I  request  to  be  able  to  study  five  different  overhaul 
manuals  at  your  location  at  a  time  and  in  a  place  agreeable 
to  you.  I  will  be  applying  readability  formulae  to  some 
randomly  selected  pages  in  each  of  these  manuals. 

4.  I  thank  you  in  advance  for  your  time  and  cooperation  to 
assist  me  in  this  research. 


Mark  Newell 
Cpt  0D 
Project  Officer 


13  Mar  1990 


SUBJECT:  Request  for  assistance  with  commercial  vs  military 

manuals 

TO:  Commander,  USA  Communications  Electronics  CECOK 

Attn:  AMSEL-LC-ME-P  <Mr.  DiMarco) 

Fort  Monmouth,  NJ  07703-5000 

1.  The  Army  Logistics  Management  Cocllege  in  conjunction 
with  the  Florida  Institute  of  Technology  is  sponsoring  a 
study  to  suggest  ways  to  improve  the  Army  equipment  overhaul 
manuals,  called  Depot  Maintenance  Work  Requirements  (DMVR> . 
This  letter  requests  copies  of  selected  manuals,  or  copies 
of  these  manuals  to  be  loaned  for  use  in  this  research. 

2.  I  am  the  principal  researcher.  My  name  is  Captain  Mark 
Newell,  SSN  550-25-9081.  I  am  stationed  at  Ft.  Ben 
Harrison,  IN,  with  duty  assignment  at  Ft.  Lee,  VA,  until  20 
June  90.  My  mailing  address  is  363-A  Coral  Sea  Dr. ,  Ft. 

Lee,  VA,  23801.  My  phone  at  which  I  can  be  reached  is  (804) 
733-7069. 

3.  I  request  use  of  the  following  manuals,  to  be  sent  to  me 
on  oan  for  one  to  two  months: 

1.  AN/GRR-23 ,  DMVR  11-5820-805 

2.  R-104/ARN,  DMVR  11-5826-208 

3.  R-109-GRC,  DMVR  11-5820-502 

4.  R-1420/URR.  DMVR  11-5820-686 

5.  R-418/G,  DMVR  11-5820-204 

6.  R-903  (XE-3 ) PRD,  DMVR  11-5820-642 

I  have  requested  six  manuals,  however,  I  will  settle  for 
five.  Since  I  do  not  have  access  to  AMC  Pam  310-9,  Index  to 
DMVRs,  I  do  not  know  is  the  above  DMVRs  still  exist,  but  I 
have  given  you  the  old  designations  to  help  in  your  search. 

4.  I  thank  you  in  advance  for  your  time  and  cooperation  to 
assist  me  in  this  research. 


Mark  Newell 
CPT  0D 

Project  Officer 
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22  May  90 

SUBJECT:  Letter  of  appreciation  for  assistance  with 

overhaul  manual  comparison  study 

TO:  Cdr ,  USA  AVSCOM 
Attn:  AVSAV-MCTP 
4300  Goodfellow  Blvd 
St  Louis,  MO  63120-1798 

1.  Thank  you  very  much  for  supplying  the  DMVR  manuals  for 
my  research.  Your  assistance  was  timely  and  accurate,  and 
your  willingness  to  help  much  appreciated. 

2.  With  your  help,  and  the  help  of  your  staff,  I  was  able 
to  complete  my  research  far  ahead  of  schedule.  By  being 
able  to  closely  inspect  those  DMVRs,  and  compare  them  to 
comparable  commercial  overhaul  manuals,  I  was  able  to  make 
observations  about  the  useability  of  DMVRs  and  commercial 
manuals  which  have  never  before  been  done.  This  information 
is  now  available  to  policy  makers  for  use  in  making  the 
depot  repair  manuals  better. 

3.  Please  contact  me  if  I  can  be  of  assistance  to  you  in 
this  matter.  My  new  address  from  1  July  90  is: 

Commander,  Red  River  Army  Depot 
Attn:  SDSRR-MC  <Cpt  Newell) 

Texarkana.  TX'  75507-5000 


Mark  Newell 
Cpt  0D 
Project  Officer 
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22  May  90 


SUBJECT:  Letter  of  appreciation  for  assistance  with 

overhaul  manual  comparison  study 

TO:  Superior  Diesel  Service 
2106  E  Main  Street 
Richmond,  VA 

1.  Thank  you  very  much  for  supplying  the  overhaul  manuals 
for  my  research.  Your  assistance  was  timely  and  accurate, 
and  your  willingness  to  help  much  appreciated. 

2.  With  your  help,  and  the  help  of  your  staff,  I  was  able 
to  complete  my  research  far  ahead  of  schedule.  By  being 
able  to  closely  inspect  those  manuals,  and  compare  them  to 
comparable  military  overhaul  manuals,  I  was  able  to  make 
observations  about  the  useability  of  DMVRs  and  commercial 
manuals  which  have  never  before  been  done.  This  information 
is  now  available  to  policy  makers  for  use  in  making  the 
depot  repair  manuals  better. 

3.  Please  contact  me  if  I  can  be  of  assistance  to  you  in 
this  matter.  My  new  address  from  1  July  90  is: 

Commander,  Red  River  Army  Depot 
Attn:  SDSRR-MC  <Cpt  Newell) 

Texarkana,  TX  75507-5000 


Mark  Newell 
Cpt  OD 
Project  Officer 
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APPENDIX  C 

READING  GRADE  LEVEL  NOMOGRAPH 
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r* 


Figure  2*1.  Instructions  for  Using  RGL  Nomograph 

Select  a  150-word  passage. 1  Begin  with  a  paragraph 
or  section  and  continue  counting  to  the  end  of  a 
complete  sentence,  even  though  it  may  exceed  150 
words. 

STEP  1 

Figure  the  average  sentence  length.  (Divide  the  number 
of  words  by  the  number  of  sentences.) 

(150  4- 10  =  15) 

STEP  2 

Figure  the  average  number  of  syllables  in  each  word. 
(Count  the  syllables.  Divide  the  number  of  syllables  by 
the  number  of  words.) 

(300  4  150  =  2.0) 

STEP  3 

Compute  the  RGL  on  the  nomograph.  3  Connect 
WORDS  PER  SENTENCE  and  SYLLABLES  PER 
WORD  using  straight  edge.  The  point  where  the  line 
crosses  the  RGL  scale  will  show  the  reading  grade  level. 
(RGL  =  13.7) 


NOTE: 


Required  manuscript  samples 

Pages 

Minimum 

Maximum 

1  to  31 

1 

4 

3210  53 

8 

$4  to  89 

6 

9 

90  and  above 

9 

30 

2  Count  sylables  the  way  you  would  say  the  word.  Court  numbers  as 
on*  word.  Count  acronym  and  abbreviations  as  one  sylabii  unless 
they  apel  a  wad  a  mat  than  one  eylable. 

Wad  Syfabtes 


Vow* 

VMtrn*»rf 

‘eyepieces’ 

'TRADOC’ 

■SOT 

•israr 

•10-28* 


WORDS 

SIR  SIMTINCI 
*• - , - 


ss 


11 
u 
•« 
so  - 


Reading 
Grade  Level 
Nomograph 


2  RGL  equates  to  rearing  dtflculy  lave)  when  I  exceeds  17  on 
the  RGL  nomograph. 


SVLLABLCS 
SIR  WORD 


S.S 

1.7 

u 

M 

1.4 

U 

u 

1.1 

i.e 

i.e 

«.• 

1.7 

14 

1.4 

1.1 

14 

1.1 


Figure  2-1.  Rearing  grade  level  nomograph 


AR  25-30  •  UPDATE 


39 
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APPENDIX  D 

FLESCH  READING  GRADE  LEVEL  NOMOGRAPH 
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APPENDIX  E 

RAW  DATA  SUMMARY  TABLE 


-113- 


*  LEGEND 


<T=truck,  A=aircraft,  R=radio,  C=commercial ,  D=DMVR) 


T1D=DMVR  9-2815-205 
T2D=DMVR  9-2815-237 
T3D=DMVR  9-2815-213 
T4D=DMVR  9-2815-210 
T5D=DXVR  9-2815-224 

A1D=DMVR  55-2835-209 
A2D=DMVR  55-2840-106 
A3D=DMVR  55-2835-205 
A4D=DKVR  55-2840-104 
A5D=DKVR  55-2840-242 

R1D=DKVR  11-5805-722 
R2D=DKVR  11-5820-667 
R3D=DKVR  11-5820-401 
R4D=DHVR  11-5820-529-50 
R5D=DMVR  11-6625-2917-5 


TlC=Cummins  manual 
T2C=Caterplllar  manual 
T3C=Perkins  engine  manual 
T4C=Mack  truck  manual 
T5C=Detroit  Diesel  manual 

AlC=Lycoming  manual 
A2C=A1 1 ison  250-C30 
A3C=A1 1 ison  250-C28 
A4C=A1 1 ison  250-C20 
A5C=Pratt  &  Whitney  manual 

R1C=GE  manual 
R2C=Motorola  manual 
R3C=EF  Johnson  manual 
R4C=RCA  manual 
R5C=Midland  Syntech  manual 
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APPENDIX  F 

ANOVA  TABLE  FORMULAS 
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BLOCK  i 

MILITARY 

MANUAL 

COMMERCIAL 

MANUAL 

ROW  TOTALS 

ROW  MEANS  X 

Truck 

Engines 

(TE) 

s. 

___ 

S 

75  (com") 

t  s 

\* 

Aircraft 

Engines 

(AE) 

S 

A£lm  it) 

__ 

^AtCcc*) 

1  s 

X« 

Radio 

Receivers 

(RR) 

^£KL*a\l)  j 

^  S 

Column 

Totals 

^  ^  Sc a>*) 

Column  _ 
Means  X 

X,  x2 

Grand  Mean  X  =  (X,  +  X*)  /  2  Grand  Mean  =  (X^£  +  x  g  +  X^)/  3 

(columns)  (rows) 


Source  of 
variation 

Sum  of  Squares 

Degrees  of 
Freedom  (DF) 

Mean 

Square 

Test 

Statistic 

Treatments 
Military  and 
Commercial 

TSS 

! 

TMS 

F  =  TMS 

EMS 

Blocks 

TE,  AE,  RR 

BSS 

2 

BMS 

F.  -  BMS 
b  EMS 

Error 

ESS 

2 

EMS 

Total 

Total  SS 

5 

TSS  =  3  £(X  -  X>* 

v 

Total  SS  =  1  (  S  -  IT)2 

TMS  =  TSS/  1  (DF) 

BMS  =  ESS  /  2  <DF ) 


BSS  =  2  f(I-  X)  2 

ESS  =  Total  SS  -  TSS  -  BSS 

BUS  =  BSS/  2  (DF) 

Degrees  of  Freedom  for  Critical  value  (1,4) 
(from  table) 
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APPENDIX  G 
POINT  SIZE  SAMPLES 
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NT/NTA  855  C.I.D.  Engine  Shop  Manual 


pulley.  Use  the  round  end  of  a  pry  bar  to 
remove  the  seal.  Do  not  damage  tne  oore  for 
the  seal. 

Remove  the  retaining  ring  (22). 

Remove  and  discard  the  O-rmg  ;24). 

Remove  the  bearing  assembly  rrom  tne  pulley. 

a.  Remove  the  plug  (29)  from  tne  pulley. 

b.  Hold  the  pulley  in  a  vise. 

c.  Put  the  flat  end  of  a  punch  tnrougn  the 
plug  hole.  Lightly  nit  tne  ounch  with  a 


USAAVSCOM  DMWR  55-2840-104  CHAPTER  4 

b.  Connect  opposite  end  of  hose  or  tube  to  OUTLET  NO.  3  STATIC 
BYPASS  port  of  test  stand  (LTCT314). 

c.  Close  STATIC  BYPASS  vmlve. 

d.  Using  hand  pump  on  test  stand,  bleed  air  from  hose  or  tube  by 
simultaneously  applying  .hydraulic  pressure  and  loosening  plug  installed  in 
preceding  step  a.  After  ail  air  has  been  bled,  tighten  plug  securely. 

e.  Using  hand  pump,  increase  pressure  until  GAGE  NO.  1  STATIC 
PRESSURE  gage  indicates  2900  to  3100  psi. 


NOTE;  When  testing  lubrication  pressure  manifold  (26,  figure  4*63),  apply 
500  to  550  psi. 


PMVR  11-5320-529-50  3-3 

"  f.  The  oscilloscope  or  counter  shall  indicate  a  1500  Hz  tlOO  Hz  tone 
or  a  1750  Hz  -100  Hz  tone  deoending  upon  the  tone  oscillator  circuit  used  in 
a  particular  transceiver  set. 

3-11.  Receiver  Sensitivitv  and  Si  anal  Plus  Noise-to-Noise  Radio  Test  / 

a.  Connect  the  equipment  as  outlined  in  paragraph  3-1  and  as  shown  in 
figure  3-1. 

b.  Adjust  the  transceiver  for  3.7  MHz  operating  frequency. 

c.  Adjust  the  signal  generator  for  3.7  MHz  output  at  0.5  microvolt  to 
the  rf  input  of  the  transceiver. 


The  2C-IC0MS  are  self-compensated  to 
2  PPM  and  can  not  provide  <  orap<  n,.iU',n  for 

EC-ICOMs. 

'•ter.  a  Z7Z  is  u  s  e  -it  r.  a  -  -  e  state  t 
transmitter  option,  compensation,  v Ltage 
for  the  5C-ICOMs  is  supplied  from  the  +10 
Volt  regulator  IC  provided  with  the  wide 
spaced  transmitter  option. 

Oscillator  Circuit 

The  quartz  crystals  used  in  ICOMs  ex¬ 
hibit  the  traditional  "S”  curve  character¬ 
istics  of  output  frequency  versus  operating 

temperature . _  _ 


GE  MANUAL 


A  constant  bias  of  5  Volts  (provided 
from  Regulator  IC  U901  in  parallel  with 
the  compensator)  establishes  the  varactor 
capacity  at  a  constant  value  over  the  etv. 
aia-tecperat ure  range.  "ith  no  addition!', 
compensation,  all  of  the  oscillators  w ill 
provide  2  PPM  frequency  stability  from  O’; 
to  55°C  (+30°F  to  131 0 F) . 

Compensator  Circuits 

Both  the  oC-ICOMs  and  2C-ICOMS  are 
temperature  compensated  at  both  ends  of  t 
temperature  range  to  provide  instant  fre¬ 
quency  compensation.  An  equivalent  ICO* 
circuit  is  shown  in  Figure  2. 
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DMVR  55-2835-209 


part*  should  be  cleaned  only  to  the 
extent  necessary  to  detect  all  flaws 
and  laperfectlons. 

g.  Because  of  the  many  variables 
Involved  In  the  cleaning  and  Inspection 
of  turbine  engine  parts.  It  Is  Impos¬ 
sible  to  set  a  standard  for  cleaning 
that  would  apply  under  all  conditions. 
Therefore,  to  obtain  the  maximum  over¬ 
haul  capacity  while  at  the  same  time 
producing  a  quality  product,  both 
cleaning  and  Inspection  personnel  must 
exercise  good  judgment  and  common  sense 
to  avoid  unnecessary  cleaning  of  parts. 

CAPTION 


C ATERF I LL AR  MANUAL  6 1 


USING  9S9082  ENGINE  TURNING  TOOL 


FUEL  SYSTEM  ADJUSTMENTS 

Checking  Fuel  Injection  Pump  Timing: 

On  Engine  Checking  with  1P540 
Flow  Checking  Tool  Group 

Tool*  Nssdsd:  1P540  Flow  Checking  Tool  Group,  7S9892 
Adapter.  9M9268  Dial  Indicator,  3S3269  Contact  Point 
(Engines  with  Pracombustion  Chambers)  or  5P2393  Contact 
Point  (Engine*  with  Direct  Iniection),  8S2296  Rod  (from 


t  t 

b.  Electrical  wiring  -  Inspect  *4 
wiring  for  faulty  Insulation 1 
and  damaged  conductors.  In-  ■ 
spect  plugs  and  receptacles  for 
corrosion,  broken,  bent,  or 
burned  pins  and  for  distorted 
or  cracked  shells. 

c.  Housings  -  Inspect  housings  for 
cracks,  dents,  scratches,  and 
corrosion.  No  cracks  are 
allowed.  Lesser  damage  such  as 
dents,  scratches,  and  corrosion 
may  be  repaired  as  Instructed 
in  the  detailed  procedures. 

d.  Spur  and  helical  gears  -  Steps 
on  tooth  profile  generated  by 


tighten  the  7S9892  Adapter  finger  tight. 

t 

CAUTION:  Do  not  use  a  wrench  to  tighten  the 
adapter.  There  will  be  damage  to  the  nozzle  seat  if 
the  adapter  is  too  tight. 

5.  Put  the  9M9268  Dial  Indicator  in  the  adapter. 
Make  an  adjustment  to  the  dial  indicator  so 
both  pointers  are  on  “0”  (zero). 

6.  Turn  the  crankshaft  a  minimum  of  45°  in  the 
CLOCKWISE  direction  (when  seen  from  the 
flywheel  end  of  the  engine). 

7.  Turn  the  crankshaft  in  the  COUNTERCLOCK¬ 
WISE  direction  (when  seen  from  the  flywheel 
end  of  the  engine)  until  the  dial  indicator  gives 


DMWR  55-2840-242 


e.  Remove  lockwire  and  separate  spark  igniter  (8)  from  rear  of  the  engine. 

f.  Remove  ignition  exciter  (10)  from  the  gearbox  by  removing  three  nuts  (1)  and 
washers  (9).  Discard  nuts  (1,  11)  and  washers  (9,  12). 

3-51.  Anti-icing  and  Bleed  Air  Systems  Dismantling.  Dismantle  anti-icing  and  bleed 
air  systems  components  from  the  engine  as  follows: 

NOTE 

The  interstage  bleed  control  valve  shall  hereafter  be  referred 
to  as  the  bleed  valve. 

a.  Detach  RH  anti-icing  air  tube  (5,  figure  3-63)  at  one  clamping  position  by 
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5.2  CLOCK  AND  VCO  ADJUSTMENT 

a.  Connect  a  frequency  counter  to  TP1  on  the 
audio/logic  board.  Adjust  C9  for  400  kHz  ±  10 
Hz. 

b.  Connect  a  DC  voltmeter  to  TP802.  With  a  Test 
PROM  programmed  for  a  center  channel  installed, 
adjust  C911  in  the  VCO  for  5.0  volts  ±  0.1  volt. 


EF  JOHNSON 
the  center  of  the  band. 


RG-188 


-*40.2 

IN. 


saaa — — LEAVE  INSULATION 
I  I  ON  CENTER  CONDUCTOR 

— *40.2  h*— 

IN. 

TEST  PROBE 

FIGURE  5-2 _ 


5.3  RECEIVER  ALIGNMENT 


c.  Connect  an  RF  voltmeter  or  spectrum  analyzer  to 
the  probe.  Insert  the  probe  into  the  hole  adjacent 
to  the  L208  adjusting  screw.  Adjust  L207  and  L208 
for  maximum  signal  output. 


When  power  is  applied  to  the  transceiver,  temporary 
keying  transients  may  be  produced  which  could 
seriously  damage  a  signal  generator  connected  to  the 


NOTE:  Do  not  short  the  helical  with  the  probe  since 
this  will  cause  improper  tuning.  Also,  do  not  repeak 


)MVH  9-2815-213  2C5 


FUEL  PUMP 


DRIVE  COUPLING 


-/v. '  v' 


v  y  a  C/  -  J 


SECURING 
CAPSCREWS  ' 


TING  AND 
AOAPTER 


FIGURE  3-99  PUMP  MOUNTED  ON  TEST  STANO 


l.  Install  1/2  mch  I.D.  flexible  hose  (3  '  gure  3- 
97.  2C3)  from  stand  "FUEL  OUTLET”  cor- 
nector  to  stana  "FUEL  RETURN"  connector 
(figure  3-101,  2C6I. 

m.  Install  flexible  hose  16,  figure  3-97  2C3)  tc 
no.  1  accumulator  can  (figure  3- 1 01 .  2C6;  from 
stand  "LEAK  TEST"  connector. 

n.  Turn  on  fuel  neat  sv/itch  (figure  3-98,  2C4) 
observe  that  temperature  on  "FUEL  TEMPERATl  RE 
gage  is  between  80  to  100  degrees  for  testing. 

CAUTION;  Never  operate  fuel  pump  until  check  valve 
hat  been  checked  for  restrictions. 

3  89.  Fuel  Pump  Run-In. 

a.  Set  stand  motor  switch  (figure  3-98,  2C4)  to 


Allison  Gas  TUrblne 

250-C2Q  Series  Operation  and  Maintenance 


2-25.  The  modulated  start  procedure  is  the  same  as  the  automatic  start  proce¬ 
dure,  including  notes  and  cautions,  except  for  the  movement  of  the  throttle  to 
the  I OLE  detent.  (Refer  to  Automatic  Start  Mode,  para  2-23. e.).  For  a 
modulated  start  advance  the  throttle  as  follows: 

a.  Wren  the  desired  cranking  speed  is  reached,  advance  the  throt¬ 
tle  toward  IDLE  until  the  engine  lights-off.  Lightoff  is  evidenced  by  com¬ 
bustion  noise  and/or  by  an  increase  in  TOT.  From  this  point,  TOT  is  control¬ 
led  directly  by  the  throttle;  Increase  or  decrease  as  required,  by  throttle 
W)V6(H6nt  • 


DMWR  11-5820-401 


/'<  f 

3. 8.1.1.  KC  Chanqe.  When  the  KC  knob  is  rotated  in  either  direction  (cw  or r 
ccw)  to  effect  frequency  change,  the  front  panel  gear  assembly  acts  as  a  differen¬ 
tial  gear  set  and  indirectly  drives  two  Oldham- type  couplers.  These  couplers  drive 
connecting  shafts  and  levers  to  perform  various  tasks  in  several  modules.  Three 
paths  of  differentiation  take  place  vrfien  the  KC  knob  is  rotated. 

a.  The  KC  knob  drives  spur  gear  F3.  Spur  gear  F3,  in  turn,  drives  spur  gear 

”  04.  Spur  gear  04  is  connected  to  the  dial  indicator,  KC  portion,  which  is 

driven  when  spur  gear  04  is  driven. 

o.  The  KC  knob  drives  spur  gear  set  F3/F2/F1  which  is  fixed  on  the  same  shaft. 
The  F2  portion  of  the  spur  gear  set  drives  spur  gear  Gl,  which,  in  turn, 
drives  the  G-shaft.  The  G-shaft  is  connected  to  an  Oldham-type  coupler 
which  drives  a  shaft  in  the  A2000A  assembly.  This  shaft  operates  S2001A 
(Interpolation  Oscillator  crystals)  and  S2001B  (Reference  Oscillator 
crystals). 


ovcvua  'UAHUAL 

P/N  3011409 


SlfcTtON  3 

SPECige  PA^TS  CLEANING 


.  in'* 


■  OOUCTION 


dlVDLAL 


|  't^Lav  Ptortto  ^  Ul»atr»md  14  <to*  (oUowla* 
g^T« /u4  N«  mir  ^irlir-t  tl—  in  proc«4arw  ui 
0-m  tor  tPOP  mmipn  la  .«*■*■«  «ato*ato.  ItoWU- 
4  liilinwi  tram  lolc  lEOPi  aa4  parttcalara  4 
opriWima  mWoM*  *>  atoMin,'  part 
in  pvm  la  9m  nauti  Ntoaa  at  9m  mfeUa. 

ITAH0A30  C  LI  AKIN  O  INJTtUCTlOKI 


GEWZRAL 


'  Carafe  Uy  taapact  la  tartar  at  atfEtoN  to 
mm*  Nat  aa  part  eTfta  avak  or  elofe  la 
oraaaat  m  trwlti  a  pnaalhla  otaatractte  to 

«ti  riov.  \ 

(to)  Kamova  from  empaal  md  pUl  .to  aaMAAa 
maa'mto  (or  Hat  li%a  elafe)  NrOaHk  tottoa. -  \. 

it)  Htaaa  Hvna^ly  Ki  P«tro4v»  ootvaa*  \ 

AJOtloa  tot  room  tamparokio. 

1  • 

(4J  Pry  4*  elaaa.  Ary.  eam»raaa>4  at r.  >♦' 

*  '  a  . 

CAUOW  OLAX*  A* 


( continued ) 
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( 5)  Install  connecting  rod  piston  pin  bushings. 


/ 


(a)  Clamp  upper  end  of  connecting  rod  in  holder  so  bore  for 
bushings  is  alined  with  hole  in  base  of  tool  (fig.  3- 
26) . 

(b)  Start  a  new  bushing  straight  into  bore  of  connecting  red 
with  bushing  joint  towards  top  of  rod  (fig.  3-29). 

(c)  Insert  installer  J4972-2  in  bushing.  Then  insert  handle 
J1513-2  into  installer,  and  drive  bushing  in  until  flange 
of  installer  meets  surface  of  connecting  rod. 

(d)  Turn  connecting  rod  over  in  holder,  and  install  second 
bushing  in  manner  noted  in  (a),  (b),  and  (c)  above. 
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A.  The  trend  check  analysis  provides  a  method  for  the  operator  to  monitor  engine  health.  The 
trend  check  will  also  allow  the  operator  to  more  effectively  predict  when  preventative  maintenance 
is  required  and  schedule  some  maintenance  actions  that  were  formerly  unscheduled. 


B.  Allisor  strongly  encourages  all  operators  on  a  voluntary  basis  to  utilize  performance  trending 
to  supplement  their  regular  maintenance  program.  It  should  be  noted  that  the  use  of  performance 
trending  does  not  change  the  requirement  to  operate  the  engine  within  established  limits  and 
according  to  applicable  publications.  Aircraft  manufacturer  engine  trending  procedures  which 
are  published  in  the  applicable  FAA  approved  flight  manuals  can  be  used  in  lieu  of  this  manual's 
procedure  if  they  are  Allison  approved.  Refer  to  Aircraft  Manufacturer  Trend  Check-Alternate 
Procedure,  para  15.  this  section. 


OMWR  11-5820-667 


CHAPTER  3 

TEST  AND  ALINEMENT  OR  ADJUSTMENT 


Section  I,  TEST  SETUP  AND  ALINEMENT  OF  MODULES 


3-1.  General 

The  basic  tests  of  individual  modules  are  referenced  in  Test  Procedures  column  of 
table  2-2.  The  alinement  procedures  for  the  individual  modules,  using  wherever 
practicable  the  test  fixtures  referenced  in  table  2-2,  in  conjunction  with  the  test 
equipment  listed  In  the  modules  functional  capacity.  Some  modules  can  be  tested 
and  alined,  while  other  modules  can  be  tested  with  no  alinement  required. 

3-2.  Test  Setup  and  Alinement  of  Module  A11A 

au  Refer  to  Test  Procedure  SC-A-401767  and  connect  the  test  equipment  as  shown 
and  described  in  the  functional  block  diagram,  figure  1. 

b.  Replace  the  module  cover  with  the  A11A  alinement  cover  (fig.  6-1). 
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DETROIT  DIESEL  53 


2.7.1. 2 


LIMITING  SPEED  MECHANICAL  GOVERNOR 


8V  ENGINE 


The  limiting  speed  mechanical  governor,  illustrated  in 
Fig.  1.  performs  the  following  functions: 

1.  Controls  the  engine  idling  speed. 

2.  Limits  the  maximum  operating  speed  of  the  engine. 

The  double-weight  governor,  identified  by  the  letters 
D.W.-L.S.  stamped  on  the  governor  name  plate,  is 
mounted  on  the  front  end  of  the  blower  and  is  driven 


The  turbocharged  engines  use  a  starting  aid  screw 
threaded  into  the  gap  adjusting  screw.  The  starting  aid 
screw  is  threaded  in  the  low-speed  gap  adjusting  screw 
so  that  its  head  contacts  the  governor  housing  wall 
(Fig.  1).  Both  the  gap  adjusting  screw  and  the  starting 
aid  screw  have  a  nylon  locking  patch  on  the  threads  in 
place  of  lock  nuts. 

Operation 

Two  manual  controls  are  provided  on  the  governor:  a 
stop  lever  and  a  speed  control  lever.  In  the  RUN 


TARCOM  DMWR  9-2815-210 


3-5.  Temoorarv  Preservation.  / 

a .  After  Cleaning  and  Preshop  Analysis. 

(1)  Assure  that  all  openings  are  covered  to  prevent 
entry  of  water  or  other  contaminants . 

(2)  Apply  a  light  coat  of  preservative  oil  to  all  bare 
metal  surfaces . 

^  •  After _ Removal  of  Major  Assemblies  and  Disassembly  of 

Subassemblies!  "  - - - 


<MENT  INSTRUCTIONS 


MIDLAND  MANUAL 


70-341/441 


Remove  the  8  screws  securing  the  top  an 
the  programmed  E/PROM  module.  Loosen  t 
cover  and  remove  the  cover.  Turn  the  v 
Cion  and  the  squelch  control  fully  ooun 
test  set  is  used,  the  rea  5  pin  test  so 
CM101  for  transmitter  al ignment  and  the 
CM202  for  receiver  alignment.  Both  tes 
with  the  unused  socket  position  toward 
Co  the  test  pins  switch  position  underl 
Supply  power  to  the  radio  and  connect  a 
a  reduced  power  output  for  a  frequency 


d  bottom  covers .  Install 
he  2  screws  securing  the  PA 
olume  control  to  a  mid  posi- 
ter  clockwise.  If  the  70-E10 
cket  should  be  connected  to 
wh i t e  5  pin  test  socket  to 
t  sockets  should  be  connected 
the  rear  of  the  radio.  Refer 
ined  in  the  steps  below, 
wattmeter  and  dummy  load  with 
counter  and  modulation  meter. 


'RANSMITTER  ALIGNMENT 


MAIN  VCO  AND  TRANSMIT  VCO  ALIGNMENT 
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CHAPTER  4 

QUALITY  AS  SU'RaNCE / QUALITY  CONTROL  REQUIREMENTS 

4-1.  General.  The  responsible  Quality  Assurance  Activity  of  the  Contractor/Depot 
shall  perform  the  quality  control  actions  specified  herein.  The  facilities  uti¬ 
lized  to  perform  these  actions  will  meet  the  requirements  of  the  procuring  activ¬ 
ity  commodity  manager  (PA/CM).  The  PA/CM  reserves  the  right  to  perrorm  any  or 
the  inspections  specified  herein  wnen  such  inspections  are  felt  to  be  necessarv 
to  assure  that  supplies  or  services  conform  to  the  prescribed  requirements. 

4-2.  Deviations  and  Exceptions.  If  the  depot  maintenance  work  requirements  car- 


TM  11  -5805-722-34/TO  31 W2-2TSC-102 


Tael*  .1-1  Front  Panel and  Chassis  Mounted  Components— Continued 


^gignator 

Nomenclature 

Figure 

reference 

yvi/Pl 

Cable,  jack 

1  3-1 

yyj/pl 

Cable/jack 

3-1 

yygpl/J2 

Cable/jack 

3-1 

ws/J* 

Cable/jack 

3-1 

w«p»j1 

i  Cable-jack 

3-1 

W1  thru  XA4 

-  Connector 

3-1 

thru  XA7 

Connector 

3-1 

xas 

Connector 

3-1 

*D91 

Lamp  holder 

4-3 

vpSJ  thru  XDS7 

Indicator 

4-3 

%DS* 

Indicator  light 

4-3 

XDS9 

Indicator 

4-3 

Y1 

Crystal  OSC 

3-1 

Yl-Pl 

Connector 

3-1 

.  j-5.  Mechanical  Inspection 

The  mechanical  inspection  procedure  is  performed 
by  making  physical  and  visual/manual  checks  of  all 
front  panel  and  chassis- mounted  components  for 
loose  or  missing  screws,  bolts,  and  nuts  and  for 
crocks  or  breaks.  Accessibility  to  these  components 


NOTE 

It  is  necessary  to  remove  the  front  panel 
to  gain  access  to  the  rear  panel  area  of 
circuit  breaker  CBl.  Refer  to  chapter  4. 
section  IV  as  required. 

3-6.  Continuity  Checks 


E5 

—Valve  Guides  (Early  Engines) 

Engines  are  lifted  with  either  detachable  valve  guides 
or  the  valve  bores  are  machined  d  'ect  “to  the 
cylinder  head 

When  wear  takes  place  in  the  valve  sc-es  cl  c.lmder 
heads  without  detachable  valve  guides  <aues  with 
oversize  stems  should  be  fitted 

Three  service  valves  are  available  ‘cr  cc’h  inlet  and 
exhaust  with  oversize  stems  of  0  003  -n  0  0t5  in  ana 
0  030  in  (0.08.  0.38  and  0.76  mm)  respectively 


•• 

E6 

9  to  See  Fig.  E.7. 

The  maximum  wear  limits  quoted  on  Page  B.3  are  tor 
areas  in  which  the  smoke  density  regulation  do  no* 
apply. 

'he  vase  seats  "  the  cylinder  head  should  be  recon- 
onioned  by  mean;  ol  cutters  or  special. sea  grinding 


-i.  Meter  Readings 

Connect  the  portable  test  set  to  site  receiver 
"'.etering  socket  and  set  the  function  seie-ctor 
switch  to  the  RCVR  position.  Che*  .,  the  tea.  : 
readings  (with  no  carrier  signal'  wii.,j.ii- 

a;th  the  typical  readings  given  .r.  Fi.fire  ?. 

FIGURE  3. 

TYPICAL  RECEIVER  METER  READINGS 
WITH  NO  CARRIER  SIGNAL 


Selector 

Typical 

Switch 

Meter 

Position 

Readings  in  uA 

1 

0  uA 

3 

0  uA 

4 

0  uA 

5 

2  5  uA 

0 

2  5  uA 

4  55  kHz  hi 
Extern .  r  t 
Disc,  5eC. 
Disc.  .  re 

1st  O  S  e  . ;  i  a 


npi.  - 

r:  ar.c 


MOTOROLA  MANUAL  25 

•list  rib  u  Uon  of  a  receiver  is  toapplyanon  -  channel 
rt  voltage  to  the  antenna  connector  while  monitor¬ 
ing  a  selected  point  in  the  receiver  with  an  rf 
voltmeter.  The  amount  of  signal  necessary  to 
produce  a  specific  reading  on  the  voltmeter  gives 
an  indication  of  the  total  gain  between  the  antenna 
connector  and  the  point  being  monitored.  The 
amount  of  signal  required  to  produce  this  reading 
can  then  be  compared  with  the  values  tabulated  in 
Figure  9.  Any  significant  deviation  between 
measured  and  tabulated  values  indicates  that  some 
circuit  between  the  antenna  and  the  point  being 
monitored  is  not  operating  correctly.  The  faulty 
circuit  can  usually  be  isolated  after  monitoring 
several  points,  where  upon  a  visual  inspection  of 
>oldcrcd  connectors  or  a  dc  voltage  check  will 
pinpoint  the  problem  area. 

Tin  tabic  below  lists  the  rf  voltage  input  re- 
in  -  u  .i.u  n  ,i  n  ail  m -a  ui  1 00  nu  1  li  vol  ts  on  t  ho 
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4-43.  TESTING  SHUTOFF  PRESSURE. 

Fuel  Pump  Testing. 

(1)  Install  fuel  pump  opto  tester  (table  2-1)  and  prime  pump  with  *2  diesel  fuel,  ten  stokes  maximum 
at  one  revolution  per  second. 

(2)  Pump  displacement  at  one  revolution  per  second  should  be  1.22  cubic  inches  (20  cc)  per  stroke 
minimum. 

(3)  Fuel  flow  at  ten  revolutions  per  second  and  2.0  psi  (13.8  kPa)  should  be  .578  oz  (16.4  g)  per  second 
at  30  revolutions  per  second  and  2.0  psi  (13.8  kPa)  flow  should  be  .889  oz  (25.2  g)  minimum. 

(4)  Shutoff  pressure  at  30  revolutions  per  second  should  be  7.49  psi  (57.1  kPa)  minimum  to  8  99  psi 
(62.0  kPa)  minimum  at  pump  outlet.  Pump  must  also  have  a  lift  of  12  in.  HG  (2.98  KPO)  at  325  rpm. 

4-44.  PAINTING,  REFINISHING,  AND  MARKING. 

Refer  to  para.  4-51  for  painting,  refinishing,  and  marking  instruction. 


RF  Circuits 


RCA  MANUAI 


RF  input  to  the  receiver  is  applied  to  the  antenna 
jack  1J7.  The  Receiver  casting  contains  the  3-TCXQ 
Mother  Module  (previously  discussed),  the  Helical  Reso¬ 
nator  Module,  tne  Buffer  Module,  the  Mixer/Multiplier 
Module,  and  the  14.5  MHz  IF  Module.  The  Helical 
Resonator  Module  contains  five,  highly  selective,  helical 
resonators  (1Z1  thru  1Z5!  with  cast  -m  cavities  and 
helically-wound  coils.  The  RF  signal  is  coupled  electro 
magnetically  through  the  cavity  apertures  and  applied  to 
the  1st  Mixer  stage  in  the  Mixer/Multipl.jr  Module 


1st  Mixer 

The  Mixer/Multiplier  Module  contains  the  1st 


Module,  consists  of  Noise  Amplifier  Q10,  Noise  Detector 
Q1 1 ,  and  Squelch  Gate  Q12  and  Q13.  When  no  carrier  is 
present,  the  noise  output  of  the  discriminator  circuit  is 
amplified  by  Noise  Amplifier  Q10  and  detected  by  Noise 
Detector  Q11.  The  output  of  the  Noise  Detector  gates 
the  Squelch  Gate  which,  in  turn,  cuts  off  Audio 
Amplifier  Q7  to  disable  the  receiver  audio  circuit.  When 
a  carrier  quiets  the  discriminator  the  Squelch  Gate  turns 
on  the  Audio  Amplifier  and  restores  the  receiver  audio. 


Receive i  Disable  IMuting)  1 

When  a  disable  (keying)  voltage  is  applied  to  pins 
18  or  32,  transistor  Q16  is  biased  into  conduction, 
which  triggers  squelch  gate  Q12/Q13  into  the  tight 
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DHWR  55-2835-205 


f.  Scan  bore  to  retracted  probe  position.  j  w 

g.  Remove  scanner  head  and  turbine  wheel  from  fixture. 

h.  Review  trace  for  any  obvious  indication  of  cracks.  Typical  indication  of 
a  crack  would  display  a  signai-to-noise  ratio  of  3:1. 

5-9.  Fluorescent  Penetrant  Inspection. 

a.  The  purpose  of  fluorescent  penetrant  (black  light)  inspection  Is  to  detect 
surface  defects  in  nonmagnetic  materials.  The  most  commonly  used  fluores- 
cent  penetrant  (item  22,  Appendix  C)  has  a  highly  fluorescent  oil  base 


EgESD  Lycoming 

LTl 01  MAINTENANCE  MANUAL 


(11)  Allow  engine  to  drain  completely  before  starting.  Ensure 
combustion  chamber  drain  valve  (refer  to  73-10-01,  figure  1, 
index  14)  is  not  clogged. 

(12)  Clean  or  replace  Pc  filter  or  perform  differential  pressure 
check.  (Refer  to  73-10-06,  paragraph  9.) 

C.  Daily  water  rinse  compressor  to  remove  salt  deposits. 

(l)  Except  on  engines  with  pneumatic  control  system  (Pc)  filter, 
disconnect  tube  assembly  (refer  to  73-10-01,  figure  1  index  3) 
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SECTION  V.  VALVES,  SOLENOIDS,  AND  FILTERS. 


5-94.  FUEL  DISTRIBUTION  VALVE  ASSEMBLY  (T55-L-7C  ENGINES).  (See  figure  5-31.)  If 
overhaul  is  required,  refer  to  paragraphs  5-95  through  5-99;  otherwise  clean 
jxterior  of  valve  and  perform  functional  test  in  accordance  with  paragraph  5-99. 


t-95.  DISASSEMBLY.  (See  figure  5-31.) 


a.  Remove  two  bolts  (1),  bracket  (12),  and  distributor  dump  valve  cover  (3). 


CAUTION  * 

< »»»»»»»###) 

Use  care  when  removing  valve  cover.  Cover  is 
under  spring  compression. 


b.  Remove  packing  (2),  and  plate  (4) 
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ALLISON  GAS  TURBINE 

250-C28  Series  Operation  and  Maintenance 


PARA  6.  (cont) 

D.  If  fiameout  has  oeen  experienced  as  the  possible  result  of  snow,  ice, 
or  water  ingestion  refer  to  Snow  Ingestion  Inspection,  para  2.E, 
Engine-inspection  Check. 

E.  If  the  aircraft  is  being  operated  following  an  extended  period  of 
inactivity,  refer  to  Special  Inspections,  table  603,  Engine-Inspec¬ 
tion/  Check,  for  recommended  action. 

F.  Ascertain  the  aircraft  oil  tank  is  properly  serviced. 
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4-29.  (CONT)  / 

r.  Use  growler  (item  11,  table  2-2)  and  test  armature  (17). 

s.  Apply  grease  (item  23,  appendix  C)  to  preformed  packing  (28)  and  install 
preformed  packing  on  end  plate  (26). 

t.  Apply  lubricating  oil  (item  25,  appendix  C)  to  six  capscrews  (15).  Align 
match  marks  and  install  commutator  end  frame  assembly  (16)  and  six  capscrews 
on  field  frame  (87).  Torque  capscrews  to  15  lb  ft  (20  N.m). 

u.  Align  three  leads  on  field  winding  (84)  with  holes  in  three  brush  holders 

(35)  and  install  three  screws  (14).  Torque  screws  to  21  +3  lb  in.  (2.4  +0.3 
N.m).  “  - 


APPENDIX  H 
F  DISTRIBUTION  TABLE 
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T-32  Appendix  Tables 
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APPENDIX  I 
SAMPLE  SIZE  GRAPH 
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APPEHDIX  I 
Sample  Size  Graph  * 


Samples 


Percentages 


*  At  a  greater  number  of  samples,  there  was  a 
lower  percent  difference  between  a  control 
group  of  20  pages  of  RGLs  and  any  successive 
sample,  breaking  greatest  at  about  2%. 
Therefore,  for  any  sample  over  about  14%, 
less  utility  was  realized  of  samples  of 
greater  size. 


APPENDIX  J 

USER  TEST  CHARACTERISTICS:  SECOND  VIEWING 
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Appendix  J 

Results:  Second  Viewing  of  the  User  Test 
Introduction 

The  following  data  were  collected  against  a  12  question 
user  test,  to  be  applied  to  the  30  manuals  of  this  study. 

The  questions  were  designed  to  illicit  response  from  an 
average,  inexperienced  reader,  that  would  expose  aspects  of 
the  manual  that  tended  to  motivate  or  demotivate  reading. 
Each  entry  below  is  a  response  to  different  parto  of  the 
questionnaire.  The  ’*+”  or  ”  sign  before  the  entry 
indicates  that  this  entry  had  a  positive  (or  negative) 
impact  on  the  motivation  of  the  user  to  read  the  text.  The 
numbers  after  the  manual  title  indicate  the  pages  from  which 
the  record  comes. 

Findings 

Cummins  Manual  8-14,  4-17. 

+  one  sentence,  one  action,  with  wide  space  between 
each  sentence;  easy  to  put  manual  down,  then  act, 
then  pick  back  up  and  use. 

DMVR  55-2840-104  4-1167,4-107. 

-  overdark  illustrations  were  unintelligible 

-  many  references  to  illustration’s  numbers,  but  the 
illustration  is  widely  separated  from  this  reference, 
causing  need  for  lots  of  page  flipping. 

+  step  by  step  repair  directions,  individually 
numbered. 

+  good  references  to  tool  numbers. 

-  single  column  text,  hard  to  quickly  see. 
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-  imprecise  wording  (’’suitable  long”). 

-  small  letters,  blurry  type. 

Mack  Truck  Manual  2-307,  2-368. 

+  a  photo  for  almost  every  repair  step. 

-  jargon  used  without  explanation  (”slinger”>. 

-  used  very  small  lettering. 

-  had  run-on  sentence  (poor  punctuation). 

DMVR  9-2815-210  2-4,  3-8. 

+  step  by  step  repair  directions,  individually 
numbered. 

-  nonstandard  words  in  table  heading  without 
explanation. 

GE  Manual  LB14984P,  LB14984. 

-  letters  too  small  and  sentences  too  close  together. 

-  long  confusing  sentences  with  lots  of  unexplained 
jargon  and  acronyms. 

-  too  few  illustrations. 

-  sentences  and  paragraphs  not  numbered;  easy  to  get  lost. 

-  lots  of  theory,  but  virtually  no  step-by  step 
repair  directions. 

-  annoying  text:  information  does  not  pass  the  ”so 
what”  test . 

DMVR  55-2835-209  5-2,5-85. 

-  imprecise  words  ("unnecessary,  quality,  maximum”). 

-  awkward  caution  sentence  construction. 

-  rambling  sentences,  confusing  reference. 

-  long  paragraphs  with  one  introductory  paragraph 
number  defeat  the  purpose  of  numbering  system. 

-  small  letters,  too  close  together. 
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+  double  column  paragraphs  are  good. 

Caterpillar  Manual  43,  61 

+  many  photos  to  go  with  each  repair  step. 

+  double  column,  wide  spaced  lines. 

+  step  by  step,  one  action-one  sentence. 

+  shows  tools  and  their  numbers  prior  to  section. 

+  layout  motivates  respect. 

+  information  motivates  respect. 

+  good  use  of  precise  words. 

DMVR  55-2840-242  3-117,  3-161 
+  well  designed  illustrations. 

-  faded  illustrations,  blurred  letters. 

+  step  by  step,  one  sentence,  one  action. 

-paragraph  heading  hides  in  text  body. 

-lots  of  undefined  jargon. 

-  precise  wording. 

-  single  column. 

EF  Johnson  5-1,  4-10 

+  aura  of  professionalism. 

+  individual  sentences,  step  by  step  procedure. 

+  bold,  crisp  lettering. 

+  layout  motivates  respect. 

+  general  instruction  is  clearly  separated  from  the 
specific  instruction. 

+  precise  photograph  illustrations. 

+  double  column  format. 

DMVR  9-2815-213.  2C5,  1G1 

-  faded  letters/illustrations  from  microfiche. 

-  letter  type  for  "note"  information  is  smaller  than 
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regular  type. 

+  one  repair  step,  one  sentence. 

+  double  column  style. 

+  there  is  a  highlighted  word  in  almost  every 
sentence . 

Allison  250-C20  2-19,  3-238 

-  mixes  direct  and  indirect  speech. 

+  crisp,  sharp  illustrations. 

+  step  by  step  repair  instruction. 

+  no  nonsense  aura  of  professionalism. 

-  single  column  style. 

DMVR  11-5820-401  3-15,  3-12 

-  extremely  busy  illustrations. 

-  some  sentences  begin  without  pronouns. 

-  single  column  style. 

Pratt  &  Whitney  manual  D-19,  G-2 

-  unmatched  tiny,  very  blurred  type. 

-  multiple  repair  steps  in  one  long  paragraph. 

-  text  mixes  double  and  single  column  style. 

-  illustrations  are  unintelligible. 

DMVR  9-2815-205  3-65,  2-14 

-  single  column  style. 

+  step  by  step  repair  procedure. 

+  good  tool  number  reference. 

Allison  250-C30P  72-00-00-57,  72-00-00-59 

-  single  column  style. 

+  bold,  crisp  letters. 

+  aura  of  professionalism. 

-  busy  illustrations. 
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-  many  words,  but  not  a  lot  of  repair  steps. 

DMVR  11-5820-667  3-1,  3-12 

-  "run-on”  in  sentence  construction  (twice). 

-  figure  numbered  6  in  a  chapter  numbered  3. 

-  missing  commas  in  sentences  (twice). 

-  imprecise  wording  ("acceptable  limits  as  stated). 

+  step  by  step  repair  information. 

-  single  column  style. 

Detroit  manual.  2. 1.1-4,  2. 7. 1.4-1 

+  step  by  step  directions  by  multiple  illustrations 
and  a  very  few  words. 

+  illustrations  are  precise  but  not  busy. 

+  layout  motivates  respect. 

DMVR  9-2815-210  3-7,  3-207 

-  poor  choice  of  word  (insure  not  ensure). 

-  no  reference  to  inspection  criteria. 

-  portions  of  illustrations  are  hand  drawn. 

-  use  of  imprecise  words  ("in  convenient  position”). 

-  single  column  style. 

Midland  manual  42.  17 

-  single  column  style. 

-  caution  statements  are  hidden  in  text. 

-  mixes  indirect  and  direct  language. 

-  unnecessarily  long  sentences. 

-  paragraphs  are  long  and  unnumbered. 

-  uses  far  too  few  illustrations. 

TM  11-5805-722-34  (and  DMVR)  3-2.  4-1 

+  Layout  motivates  respect. 

+  double  column  style. 
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+  step  by  step  numbered  procedures. 

+  illustrations  are  photographs. 

+  excellent  illustrations. 

+  precise  words. 

-  much  use  of  passive  construction. 

+  good  use  of  notes. 

-  DMVR  instructions  are  too  general. 

-  letter  point  size  is  small. 

Perkins  manual  E7,  unnumbered 

+  double  column  style. 

-  lettering  is  unusually  small. 

-  no  special  treatment  of  not  or  caution  data. 

-  no  separation  of  task  steps  by  numbers. 

-  imprecise  wording  ("suitably  marked"). 

-  excessive  shadow  in  photo  reduces  illustration 
value . 

DMVR  11-6625-29 17-5  5-1,  2-2 

-  type  is  blotchy  and  unimpressive. 

-  imprecise  words  ("Thoroughly  adequate”). 

-  improper  use  of  pronoun  number. 

-  incomplete  sentence. 

-  hand  drawn  table  does  not  command  respect. 

-  too  few  words  to  be  complete. 

-  single  column  style. 

Motorola  manual,  24 ,  25 

-  very  small  letters. 

+  double  column  style. 

-  long  paragraphs,  multi-step  procedures  without 
numbers. 
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+  aura  of  professionalism. 

DKVR  9-2815-237  4-79,  4-38 

-  cryptic  sentences,  meaning  is  obscured. 

-  single  column  style. 

+  letter  font  commands  respect. 

-  small  letter  size. 

-  mixes  active,  passive  sentence  structure. 

-  imprecise  language  ("acceptable  characteristics"). 
+  layout  commands  respect. 

RCA  manual  7,  13,  11 

-  manual  mixes  double  and  single  paragraph  style. 

-  extremely  small  letters. 

-  multiple  repair  steps  in  large,  unnumbered 
paragraphs. 

-  far  too  few  illustrations. 

-  'keying”  instructions  appear  outside  "keying" 
paragraph. 

-  tables  have  information  in  them  not  explained 
in  text. 

-  overall  annoying  style. 

DMVP  55-2835-203  5-39,  5-15 

+  good  reference  to  tool  numbers. 

-  single  column  style. 

+  step  by  step  directions,  numbered  sentences. 

-  imprecise  language  (use  of  "he"  without  antecedent 
dangling  modifier,  non-parallel  sentence 
construction) . 

Lvcoming  LT101  72-00-00-52.  72-00-00-2.2 

-  fragment  sentence. 


deleted  article  in  sentence. 


-  paragraph  headings  have  unnecessary  descriptors. 

-  single  column  style. 

-  imprecise  sentence  (problem  with  sentence  object). 

-  too  many  words  to  be  brief.  Same  sentence  repeated 
three  times. 

-  small  lettering. 

-  extremely  busy  illustrations. 

DMVR  55-2840-106  1  5-185,  5-873 

-  significant  amount  of  undefined  jargon. 

-  single  column  style. 

+  step  by  step,  numbered  repair  sentences, 

-  notes  do  not  stand  out  well  from  other  text. 

-  illustrations  are  faded  and  blurry. 

Allison  250-C28B  72-00-00-35,  72-00-00-36 

-  single  column  style. 

-  incorrect  word  use  (“ascertain  the  aircraft”). 

-  imprecise  notes:  inverted  meaning  in  sentence 
predicate . 

-  deletion  of  needed  comma  in  sentence. 

-  poor  pronoun  reference  in  warning  statement. 

-  illustrations  have  faded  lines. 

-  manual  does  not  command  respect. 

-  awkward,  poorly  written  sentence. 

DMVR  9-2815-224  4-47,  4-91 

+  good  use  of  reference  to  explain  jargon. 

-  single  column  style. 

+  step  by  step,  numbered  repair  steps. 

+  very  clear,  professional  looking  graphics. 

+  layout  motivates  respect. 
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APPENDIX  K 

SAMPLE  USABILITY  QUESTIONNAIRE 
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CUSTOMER  QUESTIONNAIRE 


To  the  Reader  of  This  Instruction  Manual: 

Motorola  is  engaged  in  a  continuous  program  of  improving  its  instruction  literature. 
We  believe  that  you  can  aid  us  in  this  program,  so  that  we  in  turn  can  better  help 
you  service  our  equipment.  To  foster  these  aims,  would  you  please  answer  the  fol¬ 
lowing  questions: 


»>i«t 

'roia 


SCHEMATIC  DIAGRAMS  AND  CABLING  DIAGRAMS 
'■  □  Are  accurate  and  easy  to  follow 

2.  |  |  Contain  minor  errors 

3.  |  |  Contain  major  errors 

4.  |  |  Are  difficult  to  follow 

If  you  have  checked  any  box  except  1,  please  tell  us  what  schematic  diagrams,  or 
portions  thereof,  were  at  fault,  or  enter  other  comments.* 


Fjrtt 

Fold 


TEXT 

Easy  to  follow  —  helps  to  service  equipment 
Would  like  more  information  on* 

iMfin  rl  -  -  _ 

iwBno  ““““ 

Tw" 


'•  □ 
*  □ 


3-  □ 


Some  instruction  sections  are  too  long  or  superfluous  such  as* 


■Second 

Fold 


4. 


Other  comments* 


(continued  on  reverse  side) 
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PARTS  LIST 


1. 

2. 


j  |  Are  complete  and  accurate 
f  I  Would  like  more  information  as  follows* 


ILLUSTRATIONS  IN  GENERAL 

1.  [  |  Are  complete  and  accurate 

*•  □  Want  more  illustrations  such  as* 


3.  |  j  Some  are  superfluous  such  as* 


The  name  of  my  instruction  manual  is: 


The  part  number  of  my  instruction  manual  is: 

(This  number  will  be  found  on  the  cover  or  on  the  title  page) 

My  name  is  _ _ 

Company_ - 

Address  — - : - 

Date _ _ _ _ _ — _ _ 

‘Whenever  possible,  give  complete  model  No.  of  equipment,  and  part  No.  of  diagram 
or  part  No.  of  instruction  section.  This  information  is  important. 


ENGINEERING  PUBLICATIONS  DEPT. 
MOTOROLA,  INC. 

1301  E.  Algonquin  Road 
Schaumburg,  Illinois  60172 
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APPENDIX  L 

RAW  DATA  FROM  RGL,  ILLUSTRATION  FREQUENCY 
AND  CUE  FREQUENCY  TESTS 
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CUMMINS  NH/NT/NTA  855 
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PERKINS  MANUAL 


-148- 


DETROIT  DIESEL 


TACOM  DHVR  9-2815-205 
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TACOX  DMVR  9-2815-237 
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TACOM  DKVR  9-2815-210 


word/sec 

chk 

chk 

svl/sec 

chk 

pa*e 

actual 

166 

— 

290 

1 

1-1 

151 

271 

40 

2-2 

165 

I 

315 

79 

3-1 

153 

■  '  m 

227 

118 

3-32 

157 

10 

237 

157 

3-65 

150 

18 

230 

196 

3-112 

156 

157 

13 

13 

226 

226 

235 

3-149 

160 

13 

248 

274 

3-187/ 

157 

10 

234 

313 

3-227 

158 

11 

231 

352 

3-266 

151 

15 

229 

391 

3-305 

167 

8 

257 

430 

4-23 

160 

6 

256 

469 

4-60 

158 

9 

303 

508 

5-2 

2209 

159 

3554 

word/sen 

RGL 

svl/wrd 

13.9 

8.  8 

1.61 

# 

pa*e 

Illust 

137 

514 

.  3 

Cue 

1404 

514 

2.  7 

TACOM  DKVR  9-2815-224 


word/sec 

chk 

sen/sec 

chk 

svl/sec 

chk 

pa*e 

actual 

156 

10 

270 

1 

1-1 

152 

11 

276 

44 

3-6 

164 

9 

318 

87 

3-27 

161 

11 

309 

120 

4-1 

151 

11 

236 

163 

4-41 

163 

12 

294 

206 

4-80 

153 

13 

225 

249 

4-116 

157 

157 

Br. 

9 

256 

254 

292 

4-158 

156 

304 

335 

4-197 

151 

242 

378 

4-232 

174 

260 

411 

4-273 

176 

282 

454 

4-319 

156 

9 

254 

497 

4-361 

164 

7 

218 

540 

4-416 

2234 

151 

3744 

word/sen 

RGL 

svl/wrd 

14.8 

ToTi 

1. 68 

# 

page 

#/PK 

Illust 

217 

559 

.4 

Cue 

2858 

559 

5.  1 
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LYCOMING  LT101 


ALLISON  250-C30  P,X 
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ALLISON  250-C28 


word/sec 

chk 

chk 

svl/sec 

chk 

paRe 

actual 

158 

1 

328 

1 

Intro  1 

153 

1 

239 

31 

72-00-00-31 

152 

266 

61 

72-00-00-10 7 

152 

258 

91 

72-00-00-319 

163 

13 

253 

121 

72-00-00-609 

153 

mm 

213 

151 

72-00-00-638 

157 

157 

BS 

232 

233 

181 

72-30-00-210 

159 

B3 

223 

211 

72-40-00-213 

153 

BBfe$ 

225 

241 

72-50-00-228 

155 

■ 

217 

271 

72-60-00-217 

152 

mm 

i 

228 

301 

73-10-01-202 

151 

BE# 

223 

331 

73-20-02-213 

165 

■a 

251 

361 

74-20-01-203 

162 

8 

282 

391 

77-20-01-204 

2185 

179 

3438 

word/sen 

RGL 

svl/wrd 

12.2 

8.0 

1. 57 

# 

Pa  Re 

#/PK 

Illust 

147 

395 

.  4 

Cue 

1314 

395 

3.3 

ALLISON  250-C20 


word/sec 

chk 

chk 

svl/sec 

chk 

paRe 

actual 

155 

9 

305 

1 

lx 

161 

8 

256 

35 

1-33/34 

157 

10 

263 

69 

2-26 

160 

9 

286 

103 

2-53 

151 

13 

260 

137 

3-27 

163 

21 

264 

171 

3-61 

179 

179 

8 

8 

287 

287 

205 

3-95 

153 

12 

250 

239 

3-129 

158 

8 

229 

273 

3-163/164 

150 

12 

227 

311 

3-197 

152 

18 

233 

345 

3-231 

150 

9 

210 

379 

3-265 

153 

13 

229 

413 

3-300 

158 

12 

218 

447 

3-333 

2200 

162 

3517 

word/sen 

RGL 

svl/wrd 

13.6 

8.8 

1 . 60 

# 

page 

#/PK 

Illust 

192 

448 

.  4 

Cue 

1174 

448 

2.6 
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PRATT  &  VHITNEY  PT  6A-20 


AVSCOM  DKVR  55-2835-209 


AVSCOM  DMVR  55-2840-106-1,2,3,4 


AVSCOM  DMVR  55-2835-205 
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AVSCOM  DKVR  55-2840-242 
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GE  SERIES  E  MANUAL 
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E  F  JOHHSON  MANUAL 
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MIDLAND  SYNTECH  MANUAL 


CECOM  DMVR  11-5820-401 
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CERCOK  DMVR  11-5820-529-50 
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APPENDIX  M 

EXAMPLE  OF  AMC  PAM  310-9 
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DMWR  55-2840-104  THRU  DMWR  55-2840- 1 06-4 
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APPENDIX  N 

SUMMARY  OF  POINT  SIZE  FINDINGS 
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SUMMARY  OF  POINT  SIZE  FINDINGS 


The  following  list  represents  the  data  collected  from 
the  manuals  in  respect  to  point  size.  The  standard  point 
sizes,  from  MIL-M-38784B,  are  located  in  Appendix  0. 


DEPOT  MANUAL 

8 

8 

7 

7 

7 

7 

7 

7 

7 

8 

7 

8 

7 

8 
8 

109 

Depot  average  point  size 
Commercial  average  point 


COMMERCIAL  MANUAL 

7 

7 

7 

7 

7 

6 

6 

7 

4 

7 

6 

7 

7 

7 

6 

98 

7.4 

size  6.5 
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APPENDIX  O 
TYPE  POINT  SIZE 
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I 

TY?8  VOXIfT  8188 

HEIGHT  OF  FONT 
DIMENSIONS 
INCHES 
(+.005) 

4 . 

L  -  .035 

. .  8  -  .025 

a. . . 

L  -  .055 

S  -  .050 

L  -  .090 
8  -  .055 

L  -  .095 
S  -  .060 

L  -  .105 
8  -  .065 

L  -  .125 
8  -  .075 


I*  -  MmIi  <pyir  um  ltttors  aid  mao  r  let  plus 

nd  y* 

8  -  IfMli  Ua  foUwlaf  lower  com  lottorts 
MiiiViViii  oad  s. 


FIGURE  6.  Typo  point  lilt 
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VITA 


Cpt  Mark  B.  Newell  was  born  on  23  October  1956.  He 
graduated  from  the  University  of  California,  Davis,  in  1981, 
with  a  degree  in  English  (teaching),  and  German.  Upon 
graduation,  he  was  commissioned  a  2nd  Lieutenant  in  the 
Army,  and  has  served  in  Organizational,  Intermediate,  and 
General  Support  Maintenance  levels  since  then,  in  Germany 
and  the  United  States.  In  Germany,  he  commanded  a  (formerly 
depot  level)  maintenance  facility,  the  Kaiserslautern 
Maintenance  Center,  performing  General  Support/  Theater 
Retrograde  maintenance  on  Ml  Abrahms  tanks  and  heavy  wheeled 
vehicles.  After  completion  of  masters  degree  s-udies  through 
the  Florida  Institute  of  Technology,  Cpt  Newell  will  work  as 
a  production  controller  at  Red  River  Army  Depot,  overhauling 
M2/M3  Bradley  Fighting  Vehicles. 
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Brief ing 


The  Depot  Systems  Command  (DESCQM) ,  a  division  of  the 
Army  Material  Command  <AMC> ,  has  sought  to  increase  the 
efficiency  of  the  Army  Depot  system  and  reduce  its  costs, 
and  has  targeted  its  overhaul  manuals  (the  t  waintenance 

Work  Requirements  (DMVRs))  for  termination.  I  is  DBSCOM’ s 
contention  that  the  DMVR  does  not  compare  favorably  with 
other  forms  of  technical  overhaul  manuals,  particularly  not 
with  the  best  forms  of  overhaul  manuals  of  the  commercial 
world,  and  it  wants  to  adopt  these  ‘’other”  forms.  However, 
DESCQM  has  not  specifically  stated  what  is  desirable  or 
undesirable  in  manuals,  nor  what  is  bad,  with  respect  to 
DMVR  usability,  or  what  is  good,  with  respect  o  commercial 
manual  usability. 

The  hypothesis  of  this  study  was  the  standard  ’’null 
hypothesis",  or,  "no  difference  between  the  two  sets.” 

This  study  provided  a  locus  of  verifiable  data  and 
related  research  to  act  as  a  source  to  answer  some  of  these 
questions.  It  employed  six  common  manual  usability  tests  to 
compare  a  representative  sample  of  overhaul  manuals  from  the 
commercial  and  the  Depot  sectors. 

viewgraph  1 
6  tests 

These  are  the  six  tests.  The  user  test  was  the  only 
qualitative  test,  and  the  other  5  were  quaint itative. 


viewgraph  2- 


Research  data 

The  related  research  showed  that: 

1.  The  two  sets  are  similar  enough  to  be  comparable 

2.  No  previous  test  showed  significant  differences 

3.  A  usability  comparison  was  seen  to  be  needed 

4.  Reading  Grade  Level  tests  are  not  good  for  the 
designing  stage  of  manuals,  but  can  be  goad  to  compare 

exi  -ting  texts.  A  lower  RGL  is  better,  if  the  manual  is  also 
Judged  readible  by  a  user  test. 

5.  Cues  help  readers  pick  out  of  texts  important 
meaning.  More  cues,  (up  to  a  point),  are  better. 

6.  Illustrations  help  conceptualize  information, 
but  should  have  as  much  color  and  realism  as  technically 
possible.  More  illustrations  are  better. 

7.  "Other”  represents  the  fact  that  larger  point 
sizes  are  better  than  smaller,  and  double  column  paragraph 
style  is  better  than  single. 

8.  The  Army  has  put  little  investment  in  designing 
DKVR  manuals  to  motivate  reading.  That  is,  making  the  text 
desireable  for  human  beings  to  read. 


o 


viewgraph  3 
Samples 

The  sample  was  a  big  one,  of  15  commercial  and  15  Depot 
manuals,  representing  more  than  12,500  manual  pages,  in 
three  equipment  catagories  (truck  engines,  aircraft  engines, 
and  radios) . 

viewgraph  4 
Findings  (1) 

This  is  the  first  of  two  viewgraphs  on  findings. 

The  findings  of  the  tests  indicated  that  there  are  no 
significant  differences  in  the  two  manual  sets. 

The  calculated  TVO-Way  AITOVA  test  for  blocks  and 
treatments  for  these  three  manual  usability  tests  are  all 
lower  than  the  critical  value  for  significance,  so  in  every 
case,  the  null  hypothesis  was  accepted.  There  are  no 
significant  usability  differences  between  the  two  manual 
sets,  as  judged  by  these  tests. 

viewgraph  5 
Findings  (2) 

1.  DMVFs  had  a  larger  average  point  size  than 
commercial  manuals,  but  had  a  smaller  point  size  than  the 
regulation  calls  for. 

2.  DMVRs  had  much  fewer  manuals  with  double  column, 
though  the  regulation  stipulates  that  they  should  always 


have  double  column,  unless  otherwise  stated. 

3.  The  user  test  was  the  best  loved  test  of  all  tests, 
by  the  researchers.  By  this  test,  DKVRS  come  out  a  little 
bit  ahead  and  not  so  far  behind,  but  the  difference  is  not 
greater  than  2%.  In  both  cases,  there  were  more  bad  aspects 
of  both  the  manuals  than  good  aspects. 

viewgraph  6 
Recommendations 

The  general  recommendations  are: 

1.  Somebody  else  should  try  their  hand  at  finding 

differences  between  the 

manuals. 

2.  A  cost  benefit  study  should  be  done  to  see  what 
value  is  added  to  Depots  by  the  usual  practice  of  rewriting 
commercial  manuals  to  make  them  into  DKVRS  (the  change  so 
far  has  not  made  the  manuals  significantly  different). 

3.  The  Army  should  pay  attention  to  "reading  motivators 
and  demot ivators” ,  because  no  matter  how  silly  they  may 
seem,  if  the  human  beings  that  overhaul  movers  (trucks  and 
aircraft),  shooters,  and  communicators  (radios),  don’t  read 
the  manuals,  how  good  can  the  quality  be? 

4.  The  findings  and  research  principles  in  this  stuay 
should  be  used  to  improve  DKVRs,  rather  than  scrapping  the 
S250  million  DMVR  library  in  favor  of  manuals  that  aren’t 
significantly  different. 
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SIX  TESTS 

♦REAIIINC  GltAIIE  LEVEL 
♦ILLUSTRATIONS 
♦CUE  FREQUENCIES 
♦USER  TESTS 
♦LETTER  POINT  SIZE 
♦RACE  LAYOUT  STYLE 


t  .  friT 


IfEriEESEXTATIVE  SAAPLE 


*15  DEPOT  AVANIIALS 
*15  CILM  MANUALS 
*5  CHAAWNTIES 
*12,594  SAMPLE  PAGES 


FINDINGS  (I) 
CALC  CRH 
RGL  .IHUI55  7.7 

ILL.  .413,.D2G  7.7 

CHE  04,  5.H3  7.7 


FAIL  Til  REJECT 


FISHINGS  (3) 
HAWK  C0A1AI 

FOIST 

SIZE  7.4  fi.5 


II*  #1 


CHLIIWS 

STYLE 


97  077 


IISEK 

TEST  407.  377.  (GOOD) 

007  037  (HAH) 


